How strong is CUBE Backgammon’s engine?

In a 1,000-game development test against GNU Backgammon using its 2-ply analysis setting, with the doubling cube switched off, CUBE Backgammon won 491 games and lost 509. With gammons worth two points and backgammons three, the final score was 644–628 in CUBE Backgammon’s favour.

To assess the engine’s strength, we need to look at both the points scored and the quality of its decisions. Comparing it with professional players also requires comparable testing and scoring methods.

Our recorded tests show competitive checker play against the GNU Backgammon settings we tested. They do not give CUBE Backgammon a verified professional-player rating. Here is what we measured, what a professional comparison would require, and why we are careful about the difference.

The test against GNU Backgammon

GNU Backgammon is a long-established open-source engine that we use as a reference. The 1,000-game test used seeded dice and alternating seats. With the cube off, it measured checker play, including the points won or lost through gammons and backgammons.

Fewer wins, more points

Games won

CUBE Backgammon491

GNU Backgammon509

Points scored

CUBE Backgammon644

GNU Backgammon628

The final score was 644 points to 628, or +0.016 points per game for CUBE Backgammon. That is a small observed edge, not proof that our engine was stronger.

The approximate 95% confidence interval runs from −0.068 to +0.100 points per game. Negative values would favour GNU and positive values would favour CUBE Backgammon, so the observed results do not rule out an advantage for either engine.

We also have a complete, later 1,000-game test against GNU Backgammon’s deeper 3-ply setting. CUBE Backgammon finished 28 points behind in that one. Both the reference setting and our development configuration had changed, so the two rows below are separate historical tests, not a controlled comparison of one upgrade.

Both intervals include zero. This means these tests do not establish which engine was stronger under their respective conditions. It does not establish that the engines were exactly equal, either. Proving that two engines are close enough to call equivalent requires deciding how large a difference matters and collecting enough evidence to rule it out.

These are records from development, rather than a fresh test of the complete current phone release. They tell us about checker play under those conditions; they do not measure full match play with the cube, or a win rate against professionals.

How do we measure a strong human player?

If you follow tournament backgammon, you will often hear about PR, or Performance Rating. In eXtreme Gammon, it measures the estimated cost of a player’s decisions relative to the analysis used to judge them. A lower PR means a lower estimated loss per decision, not a percentage of wrong moves or lost games.

The eXtreme Gammon manual explains the calculation. In simplified terms, it takes the average estimated loss per counted decision and scales it by 500. Checker plays and cube decisions can both contribute, and the rules for which decisions count matter.

For an established human benchmark, the Backgammon Masters Awarding Body, or BMAB, awards titles using analysed tournament matches. Here are the entry requirements for three title groups, rather than every grade within them:

EP means experience points: an 11-point match contributes 11 EP. These are not counts of individual games. BMAB’s process uses live, clocked tournament matches, designated in advance, with defined analysis and verification procedures. Players cannot simply submit their best few games. See the official grade table and assessment rules; the US Backgammon Federation also explains the title system.

That gives useful context for elite play, but it does not define the average professional. “Professional” describes a player’s activity, while these titles describe assessed performance. There is no single professional PR that every paid tournament player shares.

So what about CUBE Backgammon’s score of roughly 1.5?

A separate historical test examined our Expert checker decisions using GNU Backgammon’s 3-ply analysis as the reference. Across 80 self-play games, 3,519 decisions were scored and eight could not be scored. The mean estimated loss was about 0.00304. Multiply it by 500 and you get about 1.52.

The score of 1.52 is below the table’s Super Grandmaster threshold of 2.50, but the two figures were not calculated using the same method.

Our test covered checker play with the cube off. It used GNU Backgammon as the judge, rather than the eXtreme Gammon analysis used for BMAB. It also counted decisions differently: our test excluded forced plays, while eXtreme Gammon has additional rules for excluding obvious choices. Changing what goes into the average can change the score.

There is a further limit: the historical test harness did not reproduce every analysis path of the complete phone engine. We cannot assume that the missing calculations would necessarily improve the result. Nor can we turn this checker score into a full-match PR by attaching a separately measured cube score.

The result is useful evidence about those recorded decisions. It is not a human-comparable match PR, a BMAB title, or a current-release certification.

Even the engine doing the marking has limits

When an engine grades a move, its preferred answer is a reference, not a perfect view of the future. Two strong engines can disagree, and a deeper analysis can reverse the verdict.

Our own development records show two ways a comparison can mislead us. A network that matched GNU Backgammon more often chose worse moves in a separate test. In another experiment, supplying the wrong player to roll distorted the comparison itself. Neither result describes the strength of today’s release; both explain why we check more than one score.

GNU Backgammon’s manual discusses how analysis settings can bias an engine comparison. In its published performance study, Open Sage also shows why the choice of reference matters: an ordering under one engine’s analysis can reverse under another’s.

Careful rollouts help by replaying the disputed position many times, rather than relying on one quick assessment. They still have sampling uncertainty, and the answer depends on how the simulated players handle the remaining game. Our article about how a coach works explains that distinction.

Gerald Tesauro studied comparisons between backgammon engines and leading human players with TD-Gammon. His 1995 TD-Gammon article described tests and feedback involving leading players; his 2002 research paper described how self-taught evaluation and search developed beyond expert human play. Those findings concern the programs and tests described; they do not establish the same strength for another engine simply because it also uses a neural network.

What the recorded tests establish

In the two historical 1,000-game tests without the cube, CUBE Backgammon finished 16 points ahead and 28 points behind the GNU Backgammon settings tested. Both confidence intervals include zero, leaving the results compatible with an advantage for either engine.

We cannot responsibly turn that into “CUBE Backgammon beats a professional X% of the time”, assign it a human Elo, or rank it above named players. A stronger claim would need a fresh test of the complete release, including cube and match-score decisions, with comparable independent analysis and enough play to support the conclusion. A direct human challenge would also need clear match lengths, time limits and conditions.

For you as a player, the useful question is also what the coach helps you notice: when a racing lead changes your plan, why a hit is dangerous, or how the match score changes a take. Playing strength matters, but advice needs to be timely and understandable too. Our tournament-match experiment looks at that coaching question separately.

To see how much results can vary despite a constant advantage, our example of a player favoured to win 55% of matches calculates how often they still finish a session behind.

Notes on the recorded results

We recounted all 1,000 unique game records in each head-to-head test. The later test finished with 646 points for CUBE Backgammon and 674 against, from 497 wins and 503 losses. Both tests are from development in 2026, with different CUBE Backgammon configurations. The intervals use the sample standard deviation of each game’s points result (positive for a gain, negative for a loss) and the normal approximation, mean ± 1.96 standard errors, treating games as independent. They describe sampling uncertainty under that calculation, not every possible limitation of the experiment.

The checker test’s game-cluster bootstrap interval for mean loss was approximately 0.00243 to 0.00387. Rescaling that interval does not make the measure comparable to a human match PR. Eight unscored decisions are excluded, rather than silently counted as correct. No new games were played for this article. The aggregate results are available without publishing archived positions, model details or training data.