How strong is the engine, and how would you know?
· Marjon Cajocon
Every game app claims a strong AI. Almost none says what that means. Here is how strength is actually measured, and what the numbers are worth.
Self-play is a trap
The obvious test is to have a new version play the old one. Do that and you will find your engine improving forever, because both sides share the same blind spots. An engine that misunderstands a certain endgame will keep misunderstanding it, and both versions will walk into it equally often. The score says nothing.
Self-play is still how training data is generated — you need millions of labelled positions and there is no other source. But generating data and measuring strength are different jobs, and conflating them is the classic mistake.
What a gate actually is
A gate is a match, played under fixed conditions, that a candidate must win before it ships. Same time control, same opening set, both colours, enough games that the result is not noise.
The important part is that it is allowed to fail. A gate that always passes is decoration. In this project several trained networks were rejected by their gate and never shipped — one variant still runs its hand-crafted evaluation for exactly that reason. That is the gate working.
Why depth is a bad advertisement
"Searches 20 ply deep" sounds impressive and means very little. Depth depends entirely on how aggressively the search prunes, and an engine that prunes carelessly reaches great depths while missing things a shallower, more careful search would find. Two engines reporting the same depth can be hundreds of Elo apart.
What difficulty tiers really are
A difficulty setting is not a weaker engine. It is the same engine, deliberately constrained — less time, less depth, and at the lower tiers a chance of choosing a move that is good rather than best.
That last part matters for a game people actually enjoy. An engine made weak purely by reducing depth plays strangely: excellent for a while, then suddenly blind. Adding controlled imprecision instead produces an opponent that plays plausibly and makes the kind of mistakes a human might, which is what a beginner tier is for.
The honest summary
The engines here are strong enough that most players will not beat the top tiers, and the tiers below exist so that is not the only experience on offer. Precise Elo figures across draughts variants are hard to state meaningfully — there is no common rating pool to anchor to, the way chess has. Claiming a specific number would be inventing one.