sroelants / simbelmyne

A somewhat UCI compliant chess engine that could probably beat you.
GNU General Public License v3.0
17 stars 1 forks source link

Black/White asymmetry #118

Closed sroelants closed 7 months ago

sroelants commented 10 months ago

Still seeing a pretty significant playing strength asymmetry between black and white. Two exact copies of the engine playing against each other results in White winning 66% of the time, consistently.

This has me really stumped. There's not a whole lot of code in the engine that could even theoretically lead to an imbalance between black and white. The only thing I can reasonable think of is the PSTs, which are asymmetric between black and white's side (right?).

That still feels weird, since on every turn, the engine will evaluate as both sides during the negamax, so this should even out. The only thing I can think of there is the search duration asymmetry (even ply are typically much less costly than odd ply).

That could mean that, on average, the search is always cut short on an odd ply, and so an asymmetry/bug in the evaluation would always end white's search on white, and black's search on black. That could, in theory, manifest any asymmetry present in the eval.

sroelants commented 10 months ago

One easy test would be to swap out the PSTs with something symmetric, and see if the asymmetry persists. Something like this? https://www.chessprogramming.org/Simplified_Evaluation_Function

sroelants commented 10 months ago

Swapped out the PST's for the ones used in Blunder v6 (https://github.com/algerbrex/blunder/blob/6678e99e4d4884c89837e81a047dbccb5f3b079a/engine/evaluation.go#L33-L206) and still seeing the exact same behavior. Makes me think it's not the PSTs after all.

sroelants commented 10 months ago

Symmetrized the PSTs left-to-right, and it made everything a whole lot worse.

Maybe time to invest in a Board::mirror helper and make sure the static evaluation matches for the mirrored board across a test suite of board positions.