Some Interesting Go Statistics

This is part of an experiment I’ve been wanting to do for quite some time, I’ve always wanted to know some Go statistics. Now that I have the massive GoGoD database, I can finally try it out. In this post I will focus on only a few questions though; nonetheless, I would love to hear if people have other creative ones that could be answered with the same data.

What I did was to simply scan all of the games from the GoGoD database using Python and Regular Expressions (patterns in text) to extract the information from each SGF file. After going through all the pain of scraping the database, I found out that the database software it ships with allows for exporting all the data (except for the games themselves) through a CSV file. But when I tried that feature, it wasn’t working properly, the exporting process would always give me an error. Thus, I continued on with my code, it wasn’t a waste of time after all — I did learn many things about regular expressions anyway.

After the painstaking process of extracting the data, I finally managed to get everything into a Pandas DataFrame; it’s a bit tough on your computer since it has approximately 200 MB of data, but ok. If you get rid of the moves column, it goes down to only 13 MB. Anyway, I don’t want to make either of the tables freely available because it probably infringes a lot of the copyrights of the GoGoD database. Despite that, I can make my code available, as you can basically only try it if you have paid for the product, and you can find it in this Github Repo — if you have the know-how, try checking and reproducing my results, it’s important to have results verified in science after all… Now let’s move on to some of the statistics.

This one is not so interesting since you can just manually scrape go4go and find out who the champions are. It’s no surprise to any veteran Go player that the winners of the ranking would be Cho Chikun 9p and Cho Hunhyeon 9p. Nonetheless, here are the top 10 as of the date I bought the database, about 3 months ago:

1. Cho Chikun (2142)
2. Cho Hunhyeon (2023)
3. Lee Changho (1917)
4. Rin Kaiho (1705)
5. Hashimoto Utaro (1552)
6. Kobayashi Koichi (1427)
7. Seo Bongsoo (1367)
8. Sakata Eio (1349)
9. Kato Masao (1336)
10. Lee Sedol (1295)

I think it was in the Ogawa & Davies’s book about the yose (endgame) that I first read about this statistic, and it seems that they were pretty much on point, because in the book they estimated — probably from studies with pros — it as 220; and, from this database, it seems to be around 208. If you move forward to recent times, the average seems to have only been going up: 198 for the first 20,000 games, and upwards to 215 in the last 5,000. This increase in the length of games is probably an effect of the highly competitive environment we have today.

In their book about the endgame, that number was used to show that not only is Go a marathon, but about half of the game is spent in the yose, because the last part of the game usually ranges from 100-150 moves. Henceforth, I hope you have no excuse for neglecting the endgame.

This kind of statistic is also somewhat relevant for calculating the complexity of Go. One common estimate is to assume that, on average, Go has 5 candidate moves per turn, which yields \(5^{210} \approx 10^{146}\) plausible games. For chess, people often use \(3^{50} \approx 10^{24}\) for humans; and \(3^{80} \approx 10^{38}\) for machines.

What is the percentage of players for each rank? That’s probably the most curious statistic of this post/exercise. Since our era is much different than the past or the overall Go history, I decided to plot 3 graphs, one for the last 20,000 games (out of 94,000), one for the 20,000 games before that, and one for all of the games. You can see the results below.

It seems that nowadays a quarter of the active players is composed by 9 dan professionals, even though it’s the highest rank, which is understandable because the end of the ranking acts as a bottleneck. I would explain the higher numbers for the 1p-3p range as coming from the fact that the effort to get there is much smaller than getting to 5p — there are some automatic promotions up to a certain rank — and beyond, or at least that’s how most Go associations seem to think it should be.

As for the global results, I would say that today’s ranking system is harder? We have basically the same effect in less recent times (second graph), but the first bottleneck then was not at 1p-3p, it was around 5p, so I guess people are stagnating sooner nowadays. Is the root cause of all this too much competition? Too many pros having to quit to find other jobs? This would be a worthwhile comparison and study for the Asian Go Associations, since it would show with numbers what is going on with young professionals.

Out of all the games we have (close to 94,000), Black seems to have a slight edge, at a 51.5% win rate. But, if you check the most recent games, say, the last 5,000, the trend flips, and now White has the approximate 51.5%.

The main problem with komi or trying to correct the percentage to the perfect 50% is that, besides being inherently imbalanced, due to the unattainable 0.5 point, there might not be a perfect compensation, maybe the 19×19 board is imbalanced by definition. At any rate, this 1.5% sliver is kind of minuscule and shouldn’t change anyone’s life — except for Ke Jie 9p’s… —, I mean at least until the droids become living beings.

That’s it for me, I’m kind of out of ideas and wanting to enjoy my vacation. I hope this article will spark some new questions somewhere, just so we can explore them together.