search

LeelaChessZero / lc0

The rewritten engine, originally for tensorflow. Now all other backends have been ported here.
GNU General Public License v3.0
2.46k stars 535 forks source link

Syzygy TB extreme performance dropoff. #1272

Closed rooklift closed 4 years ago

rooklift commented 4 years ago

I have all 5 man Syzygy rtbw and rtbz installed. The following is a quite shocking speed difference between using them and not (on Lc0 0.25.1, Windows, Cuda)...

uci
setoption name SyzygyPath value C:\Users\Owner\Documents\Misc\Chess\Syzygy
ucinewgame
position fen 8/5b2/8/2N5/6k1/8/8/KR6 w - - 0 1
go nodes 4000

info depth 1 seldepth 2 time 16 nodes 2 score cp 3947 nps 285 tbhits 2 pv a1b2 f7a2
info depth 2 seldepth 3 time 46 nodes 19 score cp 1479 nps 500 tbhits 2 pv a1b2 f7g6 b1g1
info depth 3 seldepth 4 time 68 nodes 31 score cp 2290 nps 516 tbhits 5 pv a1b2 f7g6 b1g1 g4f5
info depth 4 seldepth 5 time 703 nodes 429 score cp 1982 nps 617 tbhits 5 pv a1b2 g4h4 b2c3 h4g4 c3d4
info depth 4 seldepth 6 time 1186 nodes 732 score cp 2455 nps 621 tbhits 18 pv a1b2 g4h4 b2c3 h4g4 c3d4 f7a2
info depth 4 seldepth 7 time 1717 nodes 1062 score cp 2088 nps 621 tbhits 26 pv a1b2 g4g3 b2c3 f7h5 c3d4 g3h3
info depth 4 seldepth 8 time 2956 nodes 1830 score cp 2108 nps 620 tbhits 69 pv a1b2 f7g6 b1g1 g4f5 b2c3 g6h5 c3d4 h5f7
bestmove a1b2 ponder f7g6

(restarted Lc0)

uci
ucinewgame
position fen 8/5b2/8/2N5/6k1/8/8/KR6 w - - 0 1
go nodes 4000

info depth 1 seldepth 2 time 12 nodes 9 score cp 4439 nps 2250 tbhits 0 pv b1b4 g4f5
info depth 2 seldepth 3 time 15 nodes 124 score cp 2397 nps 17714 tbhits 0 pv b1b8 f7c4 a1b2
info depth 3 seldepth 4 time 23 nodes 393 score cp 3156 nps 28071 tbhits 0 pv b1b8 f7d5 a1b2 d5b7
info depth 3 seldepth 5 time 30 nodes 715 score cp 2954 nps 32500 tbhits 0 pv b1b8 g4f5 b8f8 f5f6 a1b2
info depth 4 seldepth 6 time 42 nodes 1303 score cp 2672 nps 38323 tbhits 0 pv a1b2 f7g6 b1d1 g6f5 b2c3
info depth 4 seldepth 7 time 65 nodes 2506 score cp 2600 nps 44750 tbhits 0 pv a1b2 f7a2 b2a2 g4f5 a2b3
info depth 4 seldepth 7 time 94 nodes 4136 score cp 2950 nps 48093 tbhits 0 pv b1b8 f7h5 b8a8 g4h4 a8h8 h4g4
bestmove b1b8 ponder f7h5

Discord user Tinker confirmed seeing the same thing. Lc0 0.24.1 is no better. Crem suggested I file this here.

brianprichardson commented 4 years ago

Looked into this a bit more. With 4 pieces (no pawns) it is the same speed with and without TBs.

I also tried 6 pieces (removing the knight and adding pawns in front of each king). Again, it was the same speed.

Not sure if this means anything, but still odd.

mooskagh commented 4 years ago

Do you have any antivirus installed?

rooklift commented 4 years ago

Just whatever comes with Windows 10.

mooskagh commented 4 years ago

Reproduced on Linux. Have 700nps even on random backend with tablebases on SSD.

rooklift commented 4 years ago

Any chance we get a 0.25.2 for this? :)

Tilps commented 4 years ago

I'll note that I too had seen some strange performance behavior with play during TB positions - sometimes with 6 pieces still on board with TB on SSD. I hadn't managed to track it down as I didn't have a repro scenario as the games went past too fast :P. Thanks for tracking this down.