Part 2 · the workshop · autumn 2026

How the network learns Reversi

Part 1 told what we built: a Reversi opponent that learned everything by itself on a laptop. Reversi is the same game as the classic Othello. This is how, in four blocks. First the network: what the computer sees and what the "brain" actually is. Then the training: what the network does when it thinks, and how one round works. Then what we learned along the way. Last, how the network moves into a phone. No mathematics.

← Part 1 · What we built, and whyPart 2 · How we built the AI model for ReversiPå svenska
The network

1. What the computer sees

The network never sees a board with black and white discs. It sees four grids of ones and zeros, always from the point of view of the player to move: my discs, the opponent's discs, the squares where a move is allowed, and a helper plane of nothing but ones that makes it easier to sense the edge of the board. Colour does not matter. The same network plays black and white, because all it knows is "mine" and "the opponent's".

white to move mine the opponent's legal moves all ones 4 planes × 8 × 8 = 256 numbers into the network
From board to numbers. The position after black's first move, white to move. White has one disc, black four, and white may play on three squares. The four planes are all the network is told. The square a1 is always the first number and h8 the last, so the network learns the geometry, not the colours.

Two answers come back out of the network. One is a probability for every square: how likely it is that a strong player moves exactly there. We call it the policy, the gut feeling. The other is a single number between minus one and plus one: how well the player to move is doing. It is called the value, the assessment of the position.

2. What a neural network is

A neural network is a system for calculating, loosely inspired by the brain, that finds patterns in data on its own. Instead of rules that someone has written, it learns by guessing, measuring how wrong the guess was and turning its inner connections a little in the right direction. The points that calculate sit in layers, and every connection between two points has a dial, a weight, that says how much the one should affect the other.

Input Hidden layers Output takes the numbers in every line is a dial the answer
A small network. Two numbers in, one number out, and 28 dials in between. The numbers flow from left to right, and every point adds up what it receives, weighted by its dials, and passes it on. That is all a network does when it answers.

A very simple example

Picture the small network above. It gets two numbers about a position, say how many corners I have and how many moves I can choose between, and has to guess what share of the discs will be mine when the game is over. The right answer is 57. The dials are set at random, and the first guess is 30. Now four things happen, and they are the same four things that happen in our network hundreds of thousands of times a day.

  1. The error is measured. The guess 30 is 27 too low.
  2. The blame is passed backwards. For every dial it is worked out whether it pulled the guess down or up, and by how much.
  3. The dials are turned. Every dial is turned a small step in the direction that raises the guess. Only a small step, so that the network does not overshoot.
  4. The next round. The guess becomes perhaps 44, then 51, 55, 57.
1255075100 answer 57 30 error 27 13 6 2 445155
The guess creeps towards the answer. Every arrow is one round: the error is measured, the dials are turned a small step, a new guess. The steps get shorter the closer the network comes, because the error that steers the turning gets smaller. With a higher learning rate it would have gone faster but risked jumping past 57 and swinging back and forth.

With a single position the network has only learned one answer by heart. So the same thing is done with thousands of positions in turn, and the dials end up where the error is smallest across all of them at once. Then they are set right for positions the network has never seen too, and that is the whole point.

Our Reversi network works in exactly the same way, only larger: 1.8 million dials instead of 28. It is built as a tower of layers. The first layer looks at every square together with its neighbours, and every layer after that sees a little more of the board, until the tower sees all of it. At the top it splits into two heads, one for the gut feeling and one for the assessment of the position.

4 planes8 × 8 each stem128 filters, 3 × 3 6 blocks 128 filters in each, the field of view grows policy65 numbers: one per square + pass value1 number: from −1 to +1 6 × 128 = 1.8 million dials · 12 × 128 = 3.6 million
A tower with two heads. The same tower works out both the gut feeling for the next move and the assessment of the position. Six blocks of 128 filters each give 1.8 million dials.
The training

3. Thinking ahead: the search

The recipe we follow is called AlphaZero and comes from DeepMind. Its predecessor AlphaGo beat the world's best Go player in 2016, but had first been allowed to study human games. The year after, the same team showed that this was not needed: only the rules, and play against itself. AlphaZero used about 5,000 specially built chips. We used a laptop, and Reversi is a smaller game than chess.

The most important idea in the recipe is the search. A network that only follows its gut feeling makes silly mistakes, because it does not calculate. So the search tries moves in its head before it chooses: the network points out the moves it believes in, the search plays one of them, asks the network how things look now, and carries the answer back. After 200 such trial moves, the move that has been visited most often is played. Moves that look good get more visits, but untried moves get a chance too.

the position d6 net 62% · visits 121 f6 net 25% · visits 58 f4 net 13% · visits 21 black's reply…… …… … played: most visits
200 trial moves, then a real one. The network's first gut feeling is shown in percent, the search's conclusion in number of visits. Often they agree, as here. When they do not, the search has found something the network missed, and those are exactly the moments the training lives on.
The search is the teacher. The visits after 200 trial moves are a wiser opinion than the network's first gut feeling. During training the network is given exactly those as the right answer: "this is how you should have felt from the start". So the network chases a wiser version of itself, and when it has caught up, the search becomes wiser again. That is the spiral that means it never needs a human teacher.

4. The training loop

The training is a loop that repeats around the clock, one round per generation.

1 · Play2,000 games vs itself6 processes × 128 games 2 · Remember1 million positions≈ the last 8 rounds 3 · Learn950 steps of 512 positionseach position ~4 times 4 · Duel200 games vs the bestover 55% = new best 5 · Measurerandom and greedy every round,Egaroucid and the anchor every fifth next generation, the new network plays one round: ~11 min play + 2 learning + 2 duel + 1–2 measuring ≈ 16 min (large net), 12 (small)
One round, one generation. Playing is where all the data is born. Learning is the only step that turns the dials. The duel decides which network gets to play the next round's games.
  1. Play. The network plays 2,000 games against itself, with the search. A little randomness in the first moves makes the games differ.
  2. Remember. Every position is saved with two right answers: what the search thought of the moves, and how the game ended. Old games are overwritten, because they were played by a weaker network.
  3. Learn. The network guesses on positions from the memory and turns the dials towards the right answer, just as in the simple example.
  4. Duel. The new version meets the best one so far. If it takes more than 55 percent of the points, it becomes the new best and plays the next round's games. We tried lowering the bar so that fresher networks would play more often, but both times the network got worse.
  5. Measure. Last, it plays a few games that are only used for the curves, among them games against Egaroucid.
What we learned

5. What it learns first

A corner in Reversi can never be flipped, so it is the safest square on the board. The square diagonally next to a corner is the classic beginner's mistake, because it leaves the corner open to the opponent. Nobody has told the network any of this. We can measure when it works it out.

After aboutTakes the corner when it canWhat it means
2,000 games27%Chaos, barely better than chance
10,000 games86%The corners discovered: the beginner's rule in its purest form
200,000 games54%Judgement instead of a rule

The corners in an hour. After a few thousand games the network takes the corner nearly every time and has stopped giving them away. It has learned the most important rule of the board without anyone mentioning that corners exist.

Then it stops taking corners. It looks like a step backwards and is the opposite. "Always take the corner" is a beginner's rule. A strong player knows that the corner is usually still there on the next move and that something else is nearly always worth more right now. The table shows where the network stopped being a beginner.

The same pattern as in people. Zenith calculates with a hand-written evaluation in which corners are worth a lot, so it takes them as soon as it can. Juno has learned to wait.

6. Measuring right was the hardest part

The network, the search and the rules were a couple of days' work. The hard part was knowing whether the network was getting better. In self-play everything is measured against something that moves: the duel compares the new version with last week's best, which is itself on the move.

The most dangerous mistake was to believe that strength is one number. We chose networks for the app in a chain: the new one beats the one before, which beat the one before that. When we finally let the two ends meet, strength turned out to go in a circle, like rock, paper, scissors. The oldest version beat the newest. Every version had become sharp against its closest relatives and narrower at the same time.

The solution was to measure against someone outside the family, in a way that leaves chance out. We took 300 openings from human tournament games and had every network play them with both colours against Egaroucid. Since nothing is drawn at random, two networks meet exactly the same games and can be compared game by game. That measure held all the way, and it is the one we used to choose the network that sits in the app.

7. Human positions, and how it ended

The circle is the weakness of self-play: the network always meets itself, and positions its own family never builds it has hardly seen. The cure was real games. Since 1977 the French Othello Federation has collected tournament games in an archive called WTHOR, 137,548 games, free for everyone.

We used it in a way that may sound backwards: we took the positions and threw away the moves. Some training games began from a position in a human game, and from there the network played on against itself. It was never told what the human played. An earlier attempt to teach a network to imitate human moves had shown why: a network that imitates can never become better than its model. The archive contributed questions, never answers.

An unexpected bonus. All 137,548 games were replayed against our own rules engine, move by move, without a single rules error. It is the hardest test the rules have had.

With the human positions, older versions as opponents and a larger memory, we started again from scratch on 18 September. The restart trained for ten days and tried five improvements, one at a time and with the yardstick decided in advance. Every improvement moved something, but usually one part of the game at the expense of another. The best version ended up just above the network in the app, but within the margin of error.

We also tried a larger network. It learned the games better but did not win more of them. That suggests it was the games that set the limit, not the size of the network, and more games take more computing power than a laptop has. That is where we stopped, with the network in the app as the best we have built.

To the phone

8. From workshop to phone

The finished network is shrunk to a format that phones can run, 3.6 megabytes, smaller than a photo. Before the file is approved, the same positions are fed through the original and the copy, and the answers have to match. Test positions travel with the file, so that the app can make the same check on every phone.

Juno in the app is only network and search, for the whole game. To begin with, an exact solver worked out the last twelve squares. Since 26 September Juno instead searches four times deeper there, and wins just as often. The wins are narrower in discs, because the network has learned to win, not to win big.

Juno levelHow the move is chosen
EasyThe gut feeling, with a lot of randomness
MediumThe gut feeling, with a little randomness and no search
HardA short search around the network
ProA longer search, and deepest at the end: the most the network can do

The code was written together with Claude, Anthropic's AI assistant, and in a fixed order. No search was built before the rules were proven free of errors, and no network was trained before the search had played a thousand games without a rules error. A network trained on a board with a rules error learns the error, and nobody sees it in the curves.

9. Glossary

Generation
One round of the training: 2,000 games, one learning pass, one duel. The number on a network is the number of rounds.
Policy
The network's gut feeling for the next move, one probability per square.
Value
The network's assessment of the position, from −1 to +1 for the player to move.
Search
The trial moves around the network, with the grand name Monte Carlo tree search. The teacher in the recipe.
Simulation
One trial move in the search.
Duel
The new version against the best one so far. Above 55 percent, it takes over.
Egaroucid
An open, world-class Reversi program by Takuto Yamana. Our outside answer key.
Juno, Zenith
Juno is the network as an opponent in the app. Zenith is the app's minimax bots, which calculate instead of feel.
Part 1: The Reversi opponent that taught itself. The recipe: Silver et al., Mastering the game of Go without human knowledge, Nature 2017, and A general reinforcement learning algorithm that masters chess, shogi and Go through self-play, Science 2018. Egaroucid: Takuto Yamana, egaroucid.nyanyan.dev.