Book for hard/misevaluated positions

[SFisGOD]

https://github.com/jhorthos/lczero-training/wiki/Value-Repair-method

They are doing something called value repair method where they use misevaluated positions for training. I wonder if we could use this idea as well for Stockfish training/tuning. Can someone help me generate a book for this?

@ssj100 I know you have been collecting some FENs for your test suite. It would be nice to include those in the book.

[Vizvezdenec]

At first we need to get trainer that surpasses or will be on par with master net. Cause tuning is more or less a workaround for us not having one.

[SFisGOD]

@Vizvezdenec Yes, that's why I'll try this for tuning. But I need help generating the book.

[Vizvezdenec]

it's pretty hard to generate full book. Because this missevaluated positions tend to be pretty rare + need to be checked with smth that evaluates them right :) And having book of let's say 1000 positions is kinda... Idk if useful at all. And even getting 1000 positions of sf missevals with proofs is kinda hard.

[Alayan-stk-2]

You don't need to have only positions that are misevaluated at high depth. Something that SF gets very wrong at depth=12 can be a training target. And you don't need to be certain that the position is actually wrongly evaluated, just some much stronger entity (SF at higher depth and/or Leela with enough nodes) that says so.

But the lack of a good normal trainer is the biggest issue.

[Mr-Twave]

Coming out of a position in a drawkiller book, strongest large lc0 nets seem to find a way to beat strongest stockfish NNUE nearly every time at longer time controls in this position as either side. Could someone else verify this? (And is this the kind of thing we are looking for?) rnb1rbk1/5ppp/pp2p3/2ppP1q1/3P4/1P1BP3/P1P4P/KNBRQ2R b - - 0 20 There are huge misevaluations coming out of this position by both stockfish classical and stockfish NNUE, and I've extensively analyzed it. Lc0 seems to handle this position with ease. https://www.chess.com/computer-chess-championship#event=ccc-blitz-championship-2020-semifinals&game=255 Edit: to clarify, the point of posting this game here is to ask the question whether this is the kind of position that we're looking for.

[QuackQuackBlah]

@SFisGOD Here is a spreadsheet with quite a few misevaluated positions from Bryan of Fishcooking (https://docs.google.com/spreadsheets/d/1JPYelBmfxLAftsdnw7HdC9T1bJWw0S9S49i8QIsoLfs/edit#gid=0). Admittedly it has not been updated in a while, but is this helpful?

[vondele]

we do use something similar for training NNUE nets, one of the large datasets has been generated from positions where the static evaluation is different from the search result (see e.g. https://drive.google.com/file/d/1seGNOqcVdvK_vPNq98j-zV3XPE5zWAeq/view?usp=sharing). Generally, for training we need literally billions of positions, so small collections are of limited value. It wouldn't be bad if someone would collect a large number of wrongly evaluated fens... maybe a github repo with a list of fens and their game outcome would be interesting.