Closed tissatussa closed 1 month ago
Such an incredible analysis, thank you!
Yes that data is really helpful, I'll try and test it tonight, my suspicion is that it's either reaching it's max search depth, and would need to otherwise see further (I'll try bumping it up) Or it could just be the data gen I was using for that version didn't have great end game positions in it. I recently made a change that I think will help with that, currently training a new net, if successful it'll be out in v1.1.1 tonight. Finally, it could easily be the pruning, since I believe the aggressive pruning is part of what gives it's playstyle, it could also be pruning branches that it shouldn't.
Such an incredible analysis, thank you!
i just decribed what i did : among the many games this one struck me, so i copied a FEN somewhere at the ending and performed that SF test.
Yes that data is really helpful.
you mean it helps you improve the "long-term vision" in general ? i doubt that, but it can be, let's say it will surprise me when it happens - again :-) anyhow, i'm glad it gave you inspiration !
I'll try and test it tonight, my suspicion is that it's either reaching it's max search depth, and would need to otherwise see further (I'll try bumping it up) Or it could just be the data gen I was using for that version didn't have great end game positions in it. I recently made a change that I think will help with that, currently training a new net, if successful it'll be out in v1.1.1 tonight. Finally, it could easily be the pruning, since I believe the aggressive pruning is part of what gives it's playstyle, it could also be pruning branches that it shouldn't.
i was testing too : i let Sapling play White in that FEN position against stronger engines and MOST OFTEN it can win but not always !? Remember eval is +6, so we could expect even a medium strong engine can always win with White, also against Big Brother .. i set 5m+3s, but normaly this TC is from start position, so now Sapling can think much longer and maybe THEN finds its way to victory ?! We have ratings for bullet, blitz, rapid and classic overall. Btw. in my tests i used v1.0.4 (256 Mb Hash, 2 threads) because that version gave us the FEN . I conclude Sapling CAN always win, even against SF 17, but only when it has enough time per move.
However, here's a game which quickly ended in a draw !? :
[Event "engine vs engine"]
[Site "Holland"]
[Date "2024.09.24"]
[Round "?"]
[White "Sapling v1.0.4 NNUE"]
[Black "Svart v6 NNUE"]
[WhiteElo "?"]
[BlackElo "3240"]
[Result "1/2-1/2"]
[FEN "r7/3b4/2pN1pk1/2B1p1p1/1PB1P2p/p1K2P1P/P5P1/8 b - - 0 1"]
[GameDuration "00:07:02"]
[TimeControl "300+3"]
1... Kh6 {-4.04/30 23s} 2. Bf7 {+6.19/31 13s} Kg7 {-4.00/30 7.8s}
3. Bb3 {+6.19/31 12s} Kh6 {-3.99/32 9.7s} 4. Kc2 {+6.19/31 12s}
Kh7 {-4.06/30 14s} 5. Bc4 {+6.19/30 12s} Rd8 {-4.09/29 11s}
6. Kc3 {+6.18/31 11s} Ra8 {-3.99/29 14s} 7. Kb3 {+6.18/32 11s}
Kg7 {-4.05/29 29s} 8. Kc2 {+6.18/33 11s} Kg6 {-3.99/31 26s}
9. Bf7+ {+6.18/30 10s} Kg7 {-4.17/28 9.5s} 10. Kc3 {+6.18/33 10s}
Kh7 {-4.22/28 5.9s} 11. Nf5 {+6.16/29 10s} Bc8 {-4.22/30 8.1s}
12. Bb3 {+6.15/29 9.8s} Bd7 {-3.72/30 8.4s} 13. Nd6 {+6.15/28 9.5s}
Kh6 {-0.03/61 14s} 14. Kd2 {+6.19/30 9.3s} Kg7 {-3.89/30 12s}
15. Bc4 {+6.17/30 9.1s} Kh7 {-3.95/30 12s} 16. Kc3 {+6.18/32 8.9s}
Kh6 {-0.07/55 17s} 17. Bf7 {+6.15/29 8.7s} Kg7 {-0.06/58 15s}
18. Bb3 {+6.16/30 8.5s} Kh6 {-0.04/60 11s, Draw by 3-fold repetition} 1/2-1/2
I don't know why Sapling v1.0.4 didn't win this one. Probably next time it will, against Svart (3240!) i mean, and this could be true for other engines also .. it's hard to image / find a way to prove a next engine version will act "better in general" and always win that position too. What do you think ?
And i have another position which is delicate. Once i wrote this Issue about it : just win on the white squares
White to move and win.
it's a rather 'technical' position, but nice, like a puzzle, far out .. human can FEEL the concerning 'pattern' and THEN prove f6 is the right continuation, but only ShashChess seems to find the key move f6 .. i also set MultiPV because without that, the engine might never imagine f6 being a 'candidate move' .. SF 17 NNUE can't find it ! Older SF 16.1 DOES find +M19 at high depth, but only when giving f6 as the solution : then SF must think as Black and only then finds the real eval value .. Maybe this is the kind of position you're lokking for when adjusting your code and training a new net .. as i wrote, i'm not into that, and let's keep it that way :-)
Honestly, i like the way you elaborate on my input, you're passionated, i like it ! What's your own rating ?
BONUS a vid from yesterday evening - sound on !
Chess : Decline the War
Recent own online chess game, 15 min + 10 sec. I played the White pieces and chose a rare variation against the Dutch Defence : 2. g4 ?! I remembered Blacks response 3...g3! is most accurate, not capturing pawn h3 but returning the gambit pawn. So my opponent didn't seem to be surprised, but i won the game.
animation image: ColdFusion music: "Bold as Love" by Jimi Hendrix (1967) 2024@ArtFlow - all rights respected
https://github.com/user-attachments/assets/6852e8e1-66e6-4873-b3f3-180193553115
[ replay game ]
[Event "LiChess Rated rapid game"]
[Site "Holland"]
[Date "2024.09.24"]
[White "tissatussa"]
[Black "pichinzenayt"]
[WhiteElo "2013"]
[BlackElo "1957"]
[TimeControl "900+10"]
[ECO "A80"]
[Opening "Dutch Defense: Krejcik Gambit"]
[Result "1-0"]
1. d4 f5 2. g4 fxg4 3. h3 g3 4. fxg3 Nf6 5. Qd3 d6 6. Bg2 g6 7. Nc3 c6 8. a4 Bg7 9. b3 Na6 10. Ba3 O-O 11. O-O-O Qc7 12. Kb1 Bd7 13. e4 Rab8 14. Nge2 b5 15. axb5 cxb5 16. Na2 Qa5 17. b4 Qc7 18. g4 Rfc8 19. g5 Nh5 20. Bc1 Qc4 21. Bf3 Nxb4 22. Nxb4 Qxb4+ 23. Qb3+ Rc4 24. e5 Qxb3+ 25. cxb3 Rcc8 26. exd6 exd6 27. Bd5+ Kh8 28. Nf4 Bf5+ 29. Ka2 Ng3 30. Rhe1 Rc2+ 31. Kb1 Re2+ 32. Ka1 Bxd4+ 33. Rxd4 Rxe1 34. Kb2 Ne2 35. Bd2 Nxd4 36. Bxe1 a5 37. Bc3 Kg7 38. h4 Kf8 39. Bxd4 b4 40. h5 Rc8 41. Bc4 Re8 42. hxg6 hxg6 43. Nd5 Re4 44. Bf6 Be6 45. Be7+ Ke8 46. Bxd6 Kd7 47. Nf6+ Kxd6 48. Nxe4+ Ke5 49. Bxe6 Kxe6 50. Kc2 Kf5 51. Kd3 Ke5 52. Ke3 Kd5 53. Nd2 Ke5 54. Nc4+ Kf5 55. Nxa5 Kxg5 56. Nc6 Kh4 57. Nxb4 g5 58. Nd3 g4 59. b4 g3 60. Nf4 Kg4 61. b5 1-0
you mean it helps you improve the "long-term vision" in general ?
It'll help me debug the issue by replaying the exact game and seeing where the search acts strangely
I conclude Sapling CAN always win, even against SF 17, but only when it has enough time per move.
Being written in dotnet it's a little harder to write highly optimized code compared to lower level languages, but there are still a few tricks i've got left so hopefully i'll be able to bring the NPS up a little in future versions
it's hard to image / find a way to prove a next engine version will act "better in general" and always win that position too. What do you think ?
This is true, all I can realistically do it play it against its past versions using a SPRT test, which will determine if it's better on average across many different scenarios. I'm partially writing the OSCCEL to add some more variation into those tests. But it is theoretically possible it'll regress in certain areas. Though I've raised that before in the SF discord and the consensus is that if it passes the SPRT test you can assert it is a legit improvement. I do however worry that some of these improvements may lead to it loosing its unique characteristics.
Great video - Love Jimi!
So you like the music .. Right ! A quick search in my memory didn't give another chess video creation with Jimi, so i picked this one :
https://github.com/user-attachments/assets/9851a77e-60f1-462b-a5b2-8b57f0df50ae
[ also read my BitChute description ]
Just gave a few of your videos a watch, cool stuff! Do you not wana host them on YouTube? Never heard of BitChute
Do you not wana host them on YouTube? Never heard of BitChute
i did host many on YT but i always use media (images and music) that i like and find, and most of those have copyright, and YT don't like that .. i was banned several times .. but i only create what i want, no intention to make money out of it .. BitChute doesn't mind the copyrights (officially they do, but they keep my videos).
i don't wanna spoil this Issue with too many vids and other stuff, but here's the first "Ginger Boom" vid from 2017 :
https://github.com/user-attachments/assets/7d506681-a906-47ba-841f-7694b8d83f8a
@Timmoth i managed to compile the newest v1.1.2 into a single Linux file with help of https://learn.microsoft.com/en-us/dotnet/core/deploying/single-file/overview?tabs=cli : i slightly adjusted the Makefile
and ./Sapling/Sapling.csproj
and executed make -j1 publish
. The binary is 70.3 Mb in size and runs fine in CuteChess GUI. I got a bunch of warnings but no errors.
Keep em coming! Cool to watch!
Brilliant! Would you mind sharing the changes you had to make to the make file out of curiosity? Was it just the target platform to Linux-64
Also yeah it's a rather large binary due to it being 'self contained' - essentially I bundle the dotnet runtime with the exe so it doesn't require it to be installed separately
Keep em coming! Cool to watch!
i'm glad you react like that .. i saw your animation scripts, i like that! you must be an artist too .. these GitHub Issues sometimes tend to sidestep a lot, adding videos and more .. i always try to post relevant info in GitHub Issues because these are an archive also - users read them, looking for answers .. but i will upload a few more nice vids here (GitHub allows max 10Mb and offers a nice mediaplayer), since you seem to like my style .. not all people can stand two images next to eachother, both moving .. to me those animations can magically match, in sound and vision, and for many years now i learn how to handle existing (?) media synchronicities when editing - receiving back-to-the-future ..
here's one of my recent own games, i played the Black pieces .. also read the description on BitChute .. White expected me to play (an early) c7-c5 but i thought it would be dangerous to open the center that way and i didn't .. also White didn't play c2-c4 himself for a long time, giving this game special tensions .. in the end i was winning, but the positions were difficult and delicate and i had little time .. it was a team event so i decided to make a draw by repetition by which we won the match :
https://github.com/user-attachments/assets/a08a0182-6995-48e9-a0e2-19ea4bbdfb49
Would you mind sharing the changes you had to make to the make file out of curiosity? Was it just the target platform to Linux-64
i will post these code changes here this afternoon.
..it's a rather large binary due to it being 'self contained' - essentially I bundle the dotnet runtime with the exe so it doesn't require it to be installed separately
i think that's a perfect setup, we can easily handle 60Mb files nowadays ..
in Makefile
i only changed OUTPUT_DIR path.
in ./Sapling/Sapling.csproj
i adjusted the PropertyGroup
, i added these 3 lines :
<PublishSingleFile>true</PublishSingleFile>
<SelfContained>true</SelfContained>
<RuntimeIdentifier>linux-x64</RuntimeIdentifier>
then i compiled by :
$ make -j1 publish
it creates 1 single file of 70 Mb.
this way i also changed the code of v1.1.3 and compiling went well again, i even got much less warnings :
$ make -j1 publish
dotnet publish Sapling/Sapling.csproj -c Release --runtime linux-x64 --self-contained /p:Optimized=true /p:DeterministicBuild=true /p:ExecutableName= -o ./out
Determining projects to restore...
Restored /home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling.Engine/Sapling.Engine.csproj (in 691 ms).
Restored /home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling/Sapling.csproj (in 691 ms).
/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling.Engine/Evaluation/NnueWeights.cs(45,20): warning CS8602: Dereference of a possibly null reference. [/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling.Engine/Sapling.Engine.csproj]
/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling.Engine/RepetitionDetector.cs(92,26): warning CS8509: The switch expression does not handle all possible values of its input type (it is not exhaustive). For example, the pattern '0' is not covered. [/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling.Engine/Sapling.Engine.csproj]
Sapling.Engine -> /home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling.Engine/bin/Release/net8.0/Sapling.Engine.dll
Successfully created package '/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling.Engine/bin/Release/SaplingEngine.1.1.0.nupkg'.
Successfully created package '/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling.Engine/bin/Release/SaplingEngine.1.1.0.symbols.nupkg'.
/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling/UciEngine.cs(28,23): warning CS8602: Dereference of a possibly null reference. [/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling/Sapling.csproj]
/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling/Program.cs(15,34): warning CS8602: Dereference of a possibly null reference. [/home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling/Sapling.csproj]
Sapling -> /home/roelof/Compiled/Sapling-Sapling-1.1.3/Sapling/bin/Release/net8.0/linux-x64/Sapling.dll
Sapling -> /home/roelof/Compiled/Sapling-Sapling-1.1.3/out/
at this moment i let this v1.1.3 play some games ..
i like that! you must be an artist too
Thank you! Far from it, but I do enjoy certain creative coding tasks! Especially geometric stuff - pretty maths!
this way i also changed the code of v1.1.3 and compiling went well again, i even got much less warnings :
Awesome! Glad it went smoothly for you!
at this moment i let this v1.1.3 play some games ..
I can't wait to hear how it goes! I'm excited to figure out what the bug was from the other issue you found, then will start on Corrhist!
While being intrigued by the strength and playing style of your Sapling NN chess engine, i let it play many 5m+3s games against other engines during the last days - see also my #4 . Sapling wins most of those battles, slowly outperforming the opponent .. often too slow, if you ask me : it loves to keep threatening and postpones taking advantages and it does this into extremes .. i've not seen such engine "behaviour" before, Sapling has a rather unique style in my eyes. At this moment i estimate its (v1.0.x) rating 3100+
Here's one game it should clearly have won, but it couldn't :
[ replay game ]
The engine didn't find a plan to convert its great advantage (+6 !) into a win and drew this game by 3-fold repetition. That's a pitty, and i let the newest Stockfish 17 (rating 3600?!) show that White can win this position and how. It's against the same opponent engine.
I let the game start after 58. Nd6 : r7/3b4/2pN1pk1/2B1p1p1/1PB1P2p/p1K2P1P/P5P1/8 b - - 0 1 Black to play.
LiChess graph of the Sapling game. At the end White is +6 (= diagram).
[ replay game ]
I have no clue how this info could help you "fix this problem", it's a position which needs "long-term vision". i'm planning a tournament with this starting position where several strong engines must prove their 'insight' : how to win with White ?