From: Zachary Vance Date: Tue, 11 Nov 2025 05:52:57 +0000 (-0500) Subject: Go analysis X-Git-Url: https://git.za3k.com/?a=commitdiff_plain;h=42b36801e497525507014e24a79b7f7b4bbb1aa2;p=blog.git Go analysis --- diff --git a/images/19x19_komi_B.png b/images/19x19_komi_B.png new file mode 100644 index 0000000..a53a135 Binary files /dev/null and b/images/19x19_komi_B.png differ diff --git a/images/19x19_neural_B.png b/images/19x19_neural_B.png new file mode 100644 index 0000000..9ad4664 Binary files /dev/null and b/images/19x19_neural_B.png differ diff --git a/images/go-sample.png b/images/go-sample.png new file mode 100644 index 0000000..a369ca1 Binary files /dev/null and b/images/go-sample.png differ diff --git a/posts/hack2025-01-go.md b/posts/hack2025-01-go.md new file mode 100644 index 0000000..0502a15 --- /dev/null +++ b/posts/hack2025-01-go.md @@ -0,0 +1,142 @@ +--- +author: admin +categories: +- Technical +date: 2025-11-01 +tags: +- go +slug: go-analysis +title: "Hack-A-Day, Day 01: Go Analysis" +--- + +The board game Go has been revolutionized in recent years by computer play. In [2016](https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol), AlphaGo beat Lee Sedol, a top Go player. This was the equivalent of what happened in Chess in 1997. + +Since then, computers have continued to outstrip human players, but we have been learning a lot from Go engines. In this article I did some investigation using [KataGo](https://github.com/lightvector/KataGo), which I understand to essentially be an open-source clone of the AlphaGo architecture. + +This article assumes familiarity with the board game. If you're not familiar, I encourage you to give it a try sometime! Find a local try, or [play online](https://online-go.com/). + +--- + +We have only one operation we can do. We can ask KataGo to analyze a position, and tell us how good that position is. That's the only operation we'll use in this article. And we're supposed to tell KataGo what the komi is. + +KataGo returns two pieces of information for a position. An estimate of the score, and a percentage win chance. + +![caption: Score is B+12, black win rate is 99.8%](go-sample.png) + +The estimate of the score is determined (according to my very poor understanding), using an estimator which looks at the board, but doesn't try any moves. This is a fast, but low-quality metric. + +On the other hand, the win rate is detemined by, simplifying some details, trying playing the game a bunch of times really fast and seeing how often black wins. It's slower, but more accurate. + +--- + +Our first question is: How much should **komi** be? + +Using only our one tool, let's figure out what KataGo thinks. + +Well, in theory, what does a "good" komi mean? It means black and white should both win about 50% of the time. So let's just guess every possible komi, and find the one with the closest to 50% win rate. + +Or, we could use the fast score estimator on an empty board with zero komi. If it thinks black is ahead by 6.0, maybe we could set komi to 6.0. + +| size | komi estimate (winrate) | komi estimate (neural) | +|------|-------------------------|------------------------| +| 3 | +14.0 | +4.4 | +| 4 | +0.5 | +2.4 | +| 5 | +25.5 | +23.3 | +| 6 | +3.0 | +4.3 | +| 7 | +8.0 | +8.7 | +| 8 | +9.0 | +6.6 | +| 9 | +6.0 | +6.0 | +| 10 | +6.0 | +5.6 | +| 11 | +6.0 | +5.5 | +| 12 | +6.0 | +5.5 | +| 13 | +6.0 | +5.6 | +| 14 | +6.0 | +5.5 | +| 15 | +6.0 | +5.7 | +| 16 | +6.0 | +5.8 | +| 17 | +6.0 | +5.9 | +| 18 | +6.0 | +6.1 | +| 19 | +6.5 | +6.2 | + +It turns out both methods give similar results. We're going to use the win rate method going forward, because in general I've been told it's more accurate for many board positions. + +In fact, we can use the same method to evaluate *any* board position accurately. We can figure out what komi would make *that* board position 50-50 for white or black to win. And then we can treat that as the "value" of the position. + +For the rest of the article, we're going to simplify, and only ask the value of board positions. We don't care which method we use, but I'll mark the fast-and-simple method as "neural", and the winrate method as "komi" or "winrate" in pictures. + +--- + +Our next question is, what are different starting moves worth? Well, let's just play every one and see what KataGo says the score is. + +![caption: estimating score by finding win rates around 50%](19x19_komi_B.png) +![caption: estimating score with the fast neural estimator](19x19_neural_B.png) + +Note that all scores are relative to +6.5 for the empty board, which is why some values are negative. + +--- + +Okay, easy enough. What about different numbers of handicap stones? Using standard placements, we get: + +| size | handicap | value estimate (winrate) | value estimate (neural) | +|------|----------|--------------------------|-------------------------| +| 19 | 1 | +6.5 | +6.2 | +| 19 | 2 | +20.0 | +19.2 | +| 19 | 3 | +32.5 | +32.5 | +| 19 | 4 | +47.5 | +46.6 | +| 19 | 5 | +59.5 | +58.3 | +| 19 | 6 | +72.5 | +71.8 | +| 19 | 7 | +86.0 | +85.1 | +| 19 | 8 | +100.5 | +100.3 | +| 19 | 9 | +115.5 | +114.7 | +| 13 | 1 | +6.0 | +5.6 | +| 13 | 2 | +19.5 | +18.6 | +| 13 | 3 | +32.5 | +30.7 | +| 13 | 4 | +48.0 | +47.4 | +| 13 | 5 | +59.0 | +58.6 | +| 13 | 6 | +75.0 | +74.5 | +| 13 | 7 | +87.0 | +84.0 | +| 13 | 8 | +100.5 | +96.1 | +| 13 | 9 | +109.5 | +102.3 | +| 9 | 1 | +6.0 | +6.0 | +| 9 | 2 | +16.0 | +16.0 | +| 9 | 3 | +27.5 | +27.1 | +| 9 | 4 | +75.0 | +53.5 | +| 9 | 5 | +74.5 | +79.0 | + +--- + +Now let's make things more spicy. I keep winning every 1-stone game, but losing every 2-stone game. I want a 1.5 stone handicap. Well we can't add fractional stones, but we can look for something worth between 6.5 and 20 points. + +Let's do the full analysis. Every possible starting board positions. Then we'll look for one that KataGo says is worth around... say, 12 points. + +Of course, we can't really analyze every board position, so I just did ones with up to 2 stones. I included ones with white stones, because why not? + +Here's what the ones with [two black stones on 19x19](https://za3k.com/go-analysis/19x19_komi_black-black.png) look like. It might take a bit to load, and you'll need to zoom in. + +--- + +The [full set](https://za3k.com/go-analysis/) of pictures is online. + +- 19x19, 1-stone positions (black) [winrate](https://za3k.com/go-analysis/19x19_komi_B.png) [neural](https://za3k.com/go-analysis/19x19_neural_B.png) +- 19x19, 1-stone positions (white) [winrate](https://za3k.com/go-analysis/19x19_komi_W.png) [neural](https://za3k.com/go-analysis/19x19_neural_W.png) +- 19x19, 2-stone positions (black-black) [winrate](https://za3k.com/go-analysis/19x19_komi_black-black.png) +- 19x19, 2-stone positions (black-white) [winrate](https://za3k.com/go-analysis/19x19_komi_black-white.png) +- 19x19, 2-stone positions (white-white) [winrate](https://za3k.com/go-analysis/19x19_komi_white-white.png) +- 19x19, positions closest to exact point values [winrate](https://za3k.com/go-analysis/19x19_point_equivalents.png) + +- 9x9, 1-stone positions (black) [winrate](https://za3k.com/go-analysis/9x9_komi_B.png) [neural](https://za3k.com/go-analysis/9x9_neural_B.png) +- 9x9, 1-stone positions (white) [winrate](https://za3k.com/go-analysis/9x9_komi_W.png) [neural](https://za3k.com/go-analysis/9x9_neural_W.png) +- 9x9, 2-stone positions (black-black) [winrate](https://za3k.com/go-analysis/9x9_komi_black-black.png) +- 9x9, 2-stone positions (black-white) [winrate](https://za3k.com/go-analysis/9x9_komi_black-white.png) +- 9x9, 2-stone positions (white-white) [winrate](https://za3k.com/go-analysis/9x9_komi_white-white.png) +- 9x9, positions closest to exact point values [winrate](https://za3k.com/go-analysis/9x9_point_equivalents.png) + +You can also get the [raw score](https://za3k.com/go-analysis/out.csv) of 2-stone (and lower) positions on 9x9 and 19x19 boards. The code to do analysis and generate the pictures is [on github](https://github.com/za3k/hack-katago), as are details on exact software settings used. + +Thanks to Google for AlphaGo, and to lightvector for Katago (and Katago support). + +--- + +Addendum. + +After doing this project, I found it had already been done (better) at [katagobooks.com](https://katagobooks.org/). Apparently what I've done is called an "opening book", even if my goal was a bit different. +