2048 reinforcement learning
One of the best 2048 players the world has ever known.
- 2026
- C, Python, PyTorch
65536 tiles made per hour
I have always been terrible at the game 2048, so I decided to train a model to beat the highest score I could find from anyone, human or AI. After researching a few competitors, namely macroxue/2048‑ai, a deep search program, Guei, Chen and Wu's n-tuple network, and game‑difficulty/2048EndgameTablebase, a deeper search backed by terabytes of endgame tables, I found the score to beat: 1,712,092, set by 2048EndgameTablebase.
I tried a few methods and settled on a two-stage n-tuple network for most of the game, since it is much faster than a deep search, then exact endgame tables for the late game, when many of the high-value tiles are locked in place. With my best player, my chances of breaking the record were about 1 in 100,000 per game. You can find much more detail on how my model works in the “Technical details” section below.
Technical details
My player consists of two different n-tuple networks, each made of 8 lookup tables over 6 cells of the board, 272 million weights each. They learned entirely from their own games through temporal-difference (TD) learning. After every move, the network's estimate for the previous board is shifted towards the points it just earned plus its estimate for the new board, meaning the value of good positions slowly trickles back through the game as it plays. Following Guei, Chen, and Wu, I started every board's evaluation at an optimistic 320,000 points to encourage exploration, and used a fixed learning rate of 0.1 across the first 90% of training, switching to temporal coherence for the last 10% to reduce noise and allow the weights to settle. It took a total of 191 million games, over 107 hours of training across both my laptop and Google Colab, for the models to finish learning the game. When playing, the first model plays up until it makes a 16384 tile, then the second one takes over. Played games use an expectimax depth-2 search to evaluate moves.
The networks were not very good at late-game play, so once the board held both a 32768 tile and a 16384 tile, the player switches to 42 GB of exact endgame tables that I built (following macroxue's approach) which assume the largest tiles form a block in the corner. If there is no match to a table, it instead uses a depth-4 expectimax search, which scores boards by how neatly the tiles descend from a corner.
After that, I focused on speed. The odds of a record come down to how many 65536 tiles the player can make per hour, and it turned out the search before the hand-over was mostly wasted: the late game finishes a board just as well no matter how it got there. Dropping that search entirely (depth 0, late 4) makes each game weaker, with 0.5% reaching 65536 instead of 1.5%, but it plays about 20 times faster, so it makes about six times as many 65536 tiles per hour. On the same machines, that's nearly four times as many as macroxue's program at its best setting. In a run of 2,200,000 games at this setting, it averaged 422,630 points, and its best game scored 1,762,868, my current record.
My fully completed player averaged 574,188 points per game and reached the 65536 tile in 1.5% of them.
And I successfully beat the record, with my best game scoring 1,762,868 points!
Then, to find more record games, I focused on speed. Dropping the search before the endgame lowers the 65536 rate from 1.5% to 0.5%, but plays nearly 20 times faster, so it makes nearly four times as many 65536 tiles per hour as macroxue's program at its best setting. 2048EndgameTablebase searches deeper than my player and will probably beat it eventually, but its games take so long that, for now, I am ahead on the number of games played.
For fun, I also added a few modes of my own.
In cool mode, the player chooses the tile that shows up, removing randomness from the game. Because of this, a fairly simple heuristic player can score 3,932,100, the highest score any game of 2048 can reach. I was inspired by the math proofs that were released by OpenAI (openai/math), and I wanted to replicate that by having Claude write a proof in Lean that this was the maximum score, which it was able to do very easily.
In evil mode, the game places each new tile wherever it hurts most. My normal player, which expects random tiles, falls apart: its median over 100 games is 2,620 points, and its largest tile never passes 512. However, if it knows to only look at worst placements and looks six moves ahead, it reaches a median of 11,412. Increasing that to 10 moves ahead, and teaching it the fixed rule the evil placer follows, it reaches a median of 145,956.
In good mode, the best possible tile for you is placed each time. My record player averages about 1.1 to 1.2 million points there and reaches 65536 in over half its games, compared with 1.5% in the normal game.
