First Improvement to the AI Agent

We tuned the thinking seat for heads-up and sat down across from it ourselves. It learned to win the small pots and lost the match on the big ones, and the ledger shows exactly where.

Since then: board pairs, position and draws are now spelled out for the seat in code, and a guard stops it folding when checking is free (Part 10).

In Robot Wars the seat played a full six-handed night, won forty-six hands out of a hundred, and finished exactly where it started. Two problems came out of that night. It called down to the river with hands that were never ahead, and it talked about position without knowing its own seat. Then we narrowed the table to two chairs and a human took the other one. In that first heads-up match, game 990541, the seat folded 51 hands before the flop and lost 106 chips. So we tuned it. Every decision now starts fresh, with no memory of earlier hands, and its instructions now say the game is heads-up: play most hands from the button and defend the big blind. Then we played the rematch. A hundred hands, blinds of one and two, one opponent, no house bots to hide behind.

A table for two

Heads-up is a different game. Every hand you are in a blind, the button changes sides every deal, and breaking even means winning half the hands. There is nobody else at the table to take the weak hands off you. Every pot is a straight contest between the two players.

By that measure the seat lost clearly. It won 39 hands of 100 against a 50-hand baseline. It started with 200 chips and finished with 72. The human in seat ten won 61 hands and the 128 chips the machine gave up, 22 more than in the first match. Last time the scoreboard flattered the seat and the money did not. This time both of them say the same thing.

The small pots came home

The surprise is where the machine did well. Eighty-five of the hundred pots stayed small, and the seat won them on balance, netting 52 chips. That is the same kind of pot it wasted chips on in Robot Wars. When it bet the flop, the human let go fifteen times without a fight. Its own folds were mostly sound. It gave up on the turn before running into a full house twice, before a set of queens, and before four kings. The tuning worked here too. It folded before the flop 26 times, down from 51 in the first match, so far fewer blinds went to waste. The old leak became the part of its game that makes money.

The big pots went the other way

Fifteen pots grew past twenty chips, and they decided the match. The seat won five of them for 174 chips and lost ten for 354. The machine kept putting chips in, or calling, with hands that could not stand that much action.

A clear pattern ran through the worst of them. The seat treated a pair on the board as if it were its own. In hand 82 it held ace-three on a board with two jacks and lost 48 chips to pocket tens, with nothing but ace high. In hand 27 its “pair of sixes” was the board’s pair, and it paid 46 to two pair. Hand 19 was the same story with a board pair of aces, for 26 chips. In hand 83 both of its pairs were on the board, and it lost 26 on a kicker. Those four hands alone cost 146 chips, more than the whole night’s loss. A pair on the board belongs to both players. The machine played it as if it had made that pair itself.

A pair on the board belongs to both players.

Some of the big losses were just poker. Top pair ran into a set in hand 20, and a king lost to an ace on the river in hand 48. We do not want the seat to fear those, and we do not count them as leaks. The board-pair hands are different. They are mistakes it makes because of how it reads the board.

We have a suspicion about why, and we need the seat’s own reasons to confirm it. The honest packet may hand the seat the engine’s own name for its hand, a label like “Pair of Aces.” That label is accurate. It just does not say that the opponent holds the same pair. A mind that trusts the label and does not look at the board would bet exactly this way. The seat gives a short reason with every move. Once those reasons are saved with the hands, they will tell us.

Position, still unseen

The Robot Wars write-up ended with a promise: hand the seat its position. The new instructions tell it to play about eight button hands in ten, and it came close, folding about three in ten. But the results by seat are still backwards. The seat lost 93 chips on the button and only 35 in the big blind, even though the button acts last after the flop. The instructions talk about the button, but we have not confirmed that the packet tells the seat when it is on the button. We check that first.

Two honest mistakes in the ledger

The ledger is allowed to go negative and hides nothing, and this time it showed two small errors of its own. In hand 69 the machine’s row says it lost, but its bankroll went up six chips, while the human’s “winner” row went down six. One of those is wrong. Four other hands (37, 39, 73 and 74) were split pots where no chips moved, yet they are recorded as one winner and one loser. That inflates the hands-won count. The review is only as honest as the ledger under it, so both get fixed.

The plan

This is the first improvement plan for the agent. Each change is small, each is aimed at one leak from this night, and each can be measured in the next heads-up match.

1. Teach it the board. Tell the seat which cards of its hand are shared, so it knows whether its pair is really its own or on the board. Measure: no big pots on the strength of a board pair.
2. Discipline in big pots. No building a big pot with one pair or less, especially when that pair is on the board. Measure: big-pot losses cut from ten to about even with big-pot wins.
3. Confirm position reaches the packet. Check that the seat is told when it is on the button, so the heads-up instructions can actually use it. Measure: the button is its best seat, not its worst.
4. Fix the ledger. Correct the hand-69 sign and record chopped pots as splits. Measure: hands won and bankroll always agree.
5. Keep the reasons with the hands. Save the seat’s one-line reason for every move alongside the hand, then play another hundred hands and read the reasons for every pot over twenty chips. That will confirm or rule out the hand-label theory.

In Robot Wars the seat won nearly half the hands and made nothing. This time it made money in the small pots and gave all of it back in the big ones. That is progress, even if the bankroll does not show it yet. The leak is narrower now: it comes down to a handful of hands and one misread card pattern. That is a problem we can fix.

The match is on record as game 990856, one human against one thinking seat, a hundred hands.

Written by Claudette, the pen name for Claude, the AI from Anthropic that helped build HoldemRobots.AI, with Kevin Swinson. It describes the project as it stood on the date above.