New xiangqi project

[huynq55]

Hi @ianfab. It's great to know your project Fairy Stockfish. However, I want to create a xiangqi variant only (not multiple variants like Fairy-Stockfish). I want to create a simple project with minimal modifications to Stockfish code, and I think Fairy-Stockfish change the source code too much. I have seen your two projects here: https://github.com/ianfab/Makruk-Stockfish and https://github.com/ianfab/Musketeer-Stockfish. However it's hard to check what you have done. Can you please create a xiangqi version from stockfish? Or can you suggest me simple steps to do it? I really want to create one because there is no good open source xiangqi engine and I want to create one. I can donate money. Hope to hear from you soon. Thanks!

[SGQuek]

To @ianfab and @huynq55. I think you may want to look into this source code released in May 2020 https://github.com/leedavid/NewGG In accordance with http://www.ggzero.cn/forum.php?mod=viewthread&tid=274 it is a xiangqi engine capable of running on common CPU hardware, yet capable of reaching superhuman level.

Upon trying to recompile it on VS2019 community, the source code seems to be outdated. But the contents may prove helpful to all members of the team to make fairy-stockfish (or a forked project dedicated to play xiangqi variant) much stronger at playing xiangqi than now.

Thank you.

[ianfab]

@huynq55 Hi. As you mentioned I previously had created multiple Stockfish forks for different variants (Makruk, Shatranj, S-Chess, Musketeer chess). However, it quickly became apparent that adding forks for new variants or even just maintaining all the existing ones became impracticable because it multiplies the required effort. That was the reason why I started Fairy-Stockfish, because despite the added complexity it is much more efficient to maintain only one fork. This project however consumes a lot of time, so I unfortunately can not create or maintain other forks as well.

From my perspective there are no really easy steps to creating a dedicated Stockfish fork, but you could of course try to take some things (like the 128 bitboard, magic bitboards for cannon, etc.) from Fairy-Stockfish. There also have been previous attempts to make a Stockfish fork for Xiangqi like https://github.com/EterCyber/Chameleon which you can have a look at (although for me this fork never ran stably). However, from my perspective although the gains from speed improvement would be big (perhaps 100-200 Elo), implementing and training NNUE for Xiangqi can potentially result in vastly bigger playing strength improvements, as my NNUE networks for mini-xiangqi indicate (see https://www.patreon.com/ianfab or https://docs.google.com/spreadsheets/d/1fgGvBKleUOI1wZZbiNhpT98qhQ7mYuQ5kar1lTQ9g3w/edit#gid=0). Therefore, currently my focus is to make the NNUE playing and training code work also for variants with board sizes >8x8, mainly to support Xiangqi.

@SGQuek Thanks for the hint. However, although the engine might be very strong, I think there is not much one can take from this for (Fairy-)Stockfish, because it is too different. In my opinion the future for a Stockfish based Xiangqi engine should be to use NNUE for evaluation.

[SGQuek]

to @ianfab. Thank you very much for your prompt reply, greatly appreciate it.

Using NNUE proves much more promising indeed. For your information, that is indeed the programming structure utilized by all the strongest Shogi engines at the present moment (most are open source too), including Suisho 2 as mentioned in the following website

https://migigyoku.com/?p=3132

From various sources across Japanese websites, the creator of Suisho 2 apparently spent around 12 months running on one(1) Ryzen-3990 like CPU to create his NNUE from scratch. I hereby mention this information for your kind reference.

Should you need more resources to train NNUE for xiangqi, as well as any other variants such as makruk, I am more than willing to contribute my workstation with Ryzen-3970X and 192GB RAM for the training.

Thank you.

[ianfab]

Thanks, although I am aware of usages of NNUE in shogi engines (mostly Stockfish forks themselves), I did not know how much time went into training the net.

Your offer is very kind, although I currently do not know how to make use of it because my training code is private for now and I do not have any distributed training set up. If you want to contribute your hardware you are however welcome to add it as a worker to variant fishtest http://www.variantfishtest.org:6543/tests, where non-NNUE code improvements for Fairy-Stockfish are tested, also see https://github.com/ianfab/fishtest/wiki.

Also feel free to join our discord https://discord.gg/FYUGgmCFB4.

[SGQuek]

To @ianfab and @huynq55. Nevertheless, there is indeed a very severe shortcoming of the current version of fairy-stockfish observed at playing xiangqi, in which I believe should be resolved in prior of NNUE implementation: fairy-stockfish was observed to deploy the chariots(車) exceedingly late in the game.

There has being an old saying in Chinese, to whom @huynq55 may have heard: "If you do not deploy your chariot in 3 steps, you will probability lose the game" ("三步不出車，下棋多会输") Although not 100% true, it does point out the importance of launching the 2 chariots into active use.

Upon matching fairy-stockfish with other top xiangqi engines, such as NewGG, such severe drawback was observed to occur at all times, as can be seen in the following match:

Fairy-Stockfish (W) vs engine (B).zip

for which the opponent's chariot was launched much earlier than that of fairy-stockfish.

[1. c4 {+0.27/20} Che7 2. Hc2 {+0.98/20 166:30} Hg7 3. Ece2 {+0.84/20} Rh9 4. Cf2 {+0.37/21} Ha7 5. Hg2 {+0.72/22} Cbc7 6. g4 {+0.80/21} Rb9 7. Cb0 {+0.54/21} Rh3 8. Ri2 {+0.47/19} Cxe3+ 9. Hxe3 {+0.64/20} Rxe3 10. Ade1 {+0.62/21} Rg3 11. Cf1 {+0.88/19} e5 12. Cg1 {+0.49/21} Rh3 13. Hf4 {+0.98/21} e4 14. Hxg6 {+0.71/20} Afe8 15. Cd0 {+0.66/18} Rg3 16. Ch1 {+0.44/20} Rb5 17. Rh2 {+0.00/20} Hc8 18. i4 {+0.18/21} e3 19. Ra2 {+0.00/25} He7 20. Rh7 {+0.00/29} c5 21. cxc5 {+0.00/27} Rxc5 22. Rd2 {+0.00/29} Ca7 23. Cd1 {+0.37/18} Rc0+ 24. Cd0 {+0.05/22} Rc3 25. Ch5 {+0.00/21} exe2 26. Exe2 {+0.00/28} Rh3 27. Cxd9 {+0.00/30} Hf5 28. Rh8 {-0.13/27} Axd9 29. Rg8 {+0.31/26} Ece7 30. Ch8 {+0.65/24} Cxa3 31. He5 {+0.00/23} He6 32. g5 {+0.00/29} Rb3 33. gxf5 {+0.00/33} Rb0+ 34. Ad0 {+0.00/40} Ca0 35. Rdd8 {+0.00/42} Rh2 36. Eg0 {+0.00/46} Rh4 37. Ee2 {+0.00/48} Rh2 38. Eg0 {+0.00/48} Rh3 39. Ee2 {+0.00/48} Rh2]

[huynq55]

Therefore, currently my focus is to make the NNUE playing and training code work also for variants with board sizes >8x8

Is it hard to do so? How can I help? :D

[huynq55]

@SGQuek i think the move is choose base on Evaluation Function. So if we can improve it (NNUE, for example), then it will deploy chariot in right time :D

[SGQuek]

[i think the move is choose base on Evaluation Function. So if we can improve it (NNUE, for example), then it will deploy chariot in right time :D] to huynq: That may work too, but I am suggesting that maybe the early deployment of chariot should be implemented in the coding prior of introducing the NNUE architecture, which will potentially save days or even weeks of computing time later on.

Thank you.

[SGQuek]

To @huynq55: Moreover, not all flaws of engines were successfully resolved by neural-network-based architecture alone. Another example being the detection of "ladder" in the game of Go, which were resorted to be implemented in the coding before neural network. Kindly refer to the following link for more information: https://github.com/lightvector/KataGo

In such case, the implementation of some of the game's basic ruling had saved tremendous time and resources in training the neural network during the later stages of development.

[ianfab]

Thanks for your feedback. Fairy-Stockfish has many (known) weaknesses in Xiangqi, which to me is no surprise, because the evaluation is not tweaked much for Xiangqi yet. Specific playing style like timing of development of the chariot is nothing that can or should be coded into the engine explicitly in my opinion, it needs to emerge from more general considerations, like piece mobility, or control of squares. Therefore, this behavior can not specifically and easily be "fixed", but such observations can be useful to direct where to try to improve the code. Usually such weaknesses just step by step disappear as the engine improves in general, with development often being guided by findings about weaknesses though.

However, I would like you to ask to separate topics to make it easier to follow. This thread is about a potential dedicated Stockfish fork for Xiangqi. If you have some findings about Fairy-Stockfish's weaknesses, please open a new issue (like https://github.com/ianfab/Fairy-Stockfish/issues?q=is%3Aissue+is%3Aopen+label%3AXiangqi) or as mentioned feel free to join the discord server, which is a good place for general discussions about this project, since "issues" here are mostly to manage very concrete feature requests and bug reports. (If discord is not accessible from your location, you can maybe use https://github.com/ianfab/Fairy-Stockfish/discussions as an alternative.)

[ETERCYBER]

@ianfab #230, although for me this fork never ran stably, What does that mean? Did you download the latest version in the release? The latest version should be stable.

Then answer why other authors are not involved in the project:

1. Almost no one in the team updates the code related to Xiangqi. Variant test will not test Xiangqi for ten thousand years
2. Bitboard speed is extremely slow, much slower than array board, which should be everything LARGEBOARD should consider, but no one actually handles it
3. Due to support a large number of variants, the structure of the code is extremely complex, pull one, and move the whole body
4. Supporting lots of variants doesn't have any advantage in any way

In fact, it's best to have a separate set for each variant, so that it's easy for everyone to modify and contribute code. Back to Xiangqi, although we know and have tested that the Xiangqi version of NNUE is much stronger than the Xiangqi version of Lc0, we will not do it. Xiangqi is complex, which is different from chess, which can be seen from the middle game and the end game.

[ianfab]

@ianfab #230, although for me this fork never ran stably, What does that mean? Did you download the latest version in the release? The latest version should be stable.

No, I have not tried the latest release. It was just that previously it did not ran stably for me, and already at that time in the readme or a comment it was indicated that the project would be discontinued, so I just assumed that there are known issues, and I added this comment to not raise wrong expectations. I should have checked the latest release, but was in a hurry when replying.

Then answer why the author of Xiangqi did not participate in this project:

1. Almost no one in the team updates the code related to Xiangqi. Variant test will not test Xiangqi for ten thousand years

The Xiangqi related code does get updated and tested, but due to supporting so many variants it might not be that regularly. The "team" currently basically is only me, therefore I unfortunately only have very limited time to improve individual variants.

2. Bitboard speed is extremely slow, much slower than array board, which should be everything LARGEBOARD should consider, but no one actually handles it

Yes, it is slow. If you have a PoC that shows that this can be easily improved, that would be very welcome. One reason to stick to bitboards is the compatibility with official Stockfish, since I regularly merge upstream changes, which is much easier when using the same kind of data structures.

3. Due to support a large number of variants, the structure of the code is extremely complex, pull one, and move the whole body

Yes, the code base is somewhat complex. I try to separate logic and concerns as much as possible, but some complexity unfortunately is hard to avoid when supporting so many features/variants.

4. Supporting lots of variants doesn't have any advantage in any way

That very much depends on the goal. If your goal is to create the strongest possible engine for one single game/variant, I perfectly agree that having a dedicated engine is the way to go. However, if your goal is to create an engine that can play a large amount of games at a very high level, creating and maintaining a large amount of separate engines is not efficient or even practicable. Since I play chess, shogi, xiangqi, and many other variants, this why a multi-variant engine appeals to me, whereas if you are only interested in a single one of those games, writing a dedicated engine of course makes much more sense.

[kappa-wingman]

Greetings.

I am not a programmer but I found a Xiangqi engine called Challenger. It looks like it is having Stockfish because I saw some file headers have the copyright statements related to Stockfish.

About that engine: Do not use the rar provided by the repo. Looks like those file are infected when I check with Virustotal. I manage to compile it with MSVC community edition.

I had started a discussion of the Chariot weakness of SF in here https://github.com/ianfab/Fairy-Stockfish/discussions/233

[gaintpd]

Though I am not a chess engine writer, I have the smoking gun to say that the NewGG is based on stockfish.

[alwey]

search code derived from ~ commit 27ba611a3da37423a3502e49beeebe11c9a11d8e

[ETERCYBER]

That very much depends on the goal. If your goal is to create the strongest possible engine for one single game/variant, I perfectly agree that having a dedicated engine is the way to go. However, if your goal is to create an engine that can play a large amount of games at a very high level, creating and maintaining a large amount of separate engines is not efficient or even practicable. Since I play chess, shogi, xiangqi, and many other variants, this why a multi-variant engine appeals to me, whereas if you are only interested in a single one of those games, writing a dedicated engine of course makes much more sense.

Whenever we write an AI engine, we always hope that the level of the engine is higher than that of human beings, and that some obvious bugs will be avoided as far as possible. The support for variants is as far as possible, rather than just making up for the number (the level of some variants is relatively weak, even far below the ordinary level of human beings, but it is claimed on the Wiki that they have surpassed the Masters), which is not in line with the design requirements The original intention.

Stockfish's success lies in constantly simplifying the code to make the project maintainable, but this project is just the opposite.

[gbtami]

"Whenever we write an AI engine, we always hope that the level of the engine is higher than that of human beings" Sometimes yes, sometimes no. See https://lichess.org/blog/X9PUixUAANCqFRSh/introducing-maia-a-human-like-neural-network-chess-engine If your goal is the same as the goal of upstream Stockfish devs, then Fairy-Stockfish is definitely not for you.

[ianfab]

@EterCyber I develop this project for fun and in the hope it is useful, and constructive criticism is always welcome. However, in my opinion there is not much point in questioning the most fundamental goal of a project. Different people have different goals, and so have different projects. And everyone has the freedom to choose the projects whose goals he shares.

If you have concrete suggestions how to improve the project, or if you have found an error in the code or the documentation, feedback is very welcome. I always try to be transparent about the project's strengths and weaknesses, but I also make mistakes, so if you have found a statement in the wiki that is not true, then please point me to the exact statement and provide evidence why it is not true, and I will happily correct it.

Whenever we write an AI engine, we always hope that the level of the engine is higher than that of human beings

No, at least not necessarily for me. Of course I hope it will, but it is not a strict requirement. My goal for an engine is to make it as strong as possible given its design constraints. Whether "as strong as possible" for a given variant then is below top human level or above it is something that time will tell. Neither do I stop when the engine's level is below human level, nor do I stop when it surpasses the level of all humans and engines, because my goal simply is to improve it as much as possible.

Stockfish's success lies in constantly simplifying the code to make the project maintainable, but this project is just the opposite.

Yes, Stockfish has a strong emphasis on code simplicity and I share this approach (after all I also contribute to official Stockfish, although rather rarely due to lack of time). Due to this project's goal and scope, the code base necessarily is much more complex than official Stockfish, which however does not mean that I do not try to simplify it as much as possible. On the contrary, I sometimes even refrain from certain small playing strength improvements in order to not further complicate the code.

In short, I always strive to improve playing strength and reduce code complexity, but the two have to be balanced against each other and against the project's scope, and it is impossible to improve all three at the same time, so I try to maintain a good balance between the three in alignment with the project's goals.

[maksimKorzh]

@huynq55 I have exactly the same ideas in mind) https://github.com/ianfab/Fairy-Stockfish/discussions/277

[maksimKorzh]

Hi @EterCyber Wilbert, I just mailed you to email specified in your github profile with an ideas for xiangqi stockfish fork. I've also started a new thread regarding this idea: https://github.com/ianfab/Fairy-Stockfish/discussions/277 Hope we can collaborate.

[gbtami]

@maksimKorzh you can post in https://github.com/EterCyber/Chameleon/issues directly :)

[maksimKorzh]

@gbtami thank you, yeah, that's really good idea))) sorry for fludding)