Support UCI-Cyclone

[HGMuller]

It turns out that Fairy-Stockfish numbers ranks 1 to 10 on boards with 10 ranks, in UCI mode. The de facto standard for this is to use 0-9. This makes SF incompatible with existing, for instance, existing UCI Xiangqi engines such as Cyclone. It also makes it impossible to run SF as UCI engine under XBoard + UCI2WB (e.g. to work around the ponder problem).

[ianfab]

Considering that at least Cutechess, Sjaak II, and Fairy-Stockfish use the more natural, in the sense of not creating an unnecessary exception, counting from 1 to 10 for the UCI protocol, while of course all use the 0 to 9 counting in CECP, I question the existence of a de facto standard here. CECP and UCCI decided to use the 0 to 9 counting and that is all fine, but introducing it in UCI just because a specific engine for a specific variant uses it (although it would be much better off just using UCCI, but I admittedly do not know if Cyclone predates UCCI) would be confusing. Since Fairy-Stockfish also supports UCCI, you can also try UCCI2WB if you do not want to change the move conversion in UCI2WB due to compatibility with existing Xiangqi engines.

[HGMuller]

I was not aware Sjaak II used 1-10 counting in UCI mode, or I would certainly have taken it up with the author. I had assumed it would follow Cyclone, by then the only UCI engine for a variant larger than 8x8. It seems that this incompatibility let the genie out of the bottle. But there never was any need for me to run Sjaak II as UCI, as it doesn't have a ponder problem in CECP, and Evert apparently did not feel any need to test his UCI Xianggi implementation under WinBoard.

Cyclone UCI probably indeed predates UCCI. But it used to be one of the strongest Xiangqi engines around. And the accompanying GUI was very popular in China.

If UCCI mode in Fairy-Stockfish also works for other variants than Xiangqi, the damage still is limited, however. In particular, those who want to use it for Janggi can use it with UCI2WB forced to UCCI mode. [Edit] I guess this doesn't work, because the UCI_Variants option is not recognized in UCCI; it is a UCI-specific option.

[alwey]

I would recommend sticking to the UCI spec

The move format is in long algebraic notation. A nullmove from the Engine to the GUI should be send as 0000. Examples: e2e4, e7e5, e1g1 (white short castling), e7e8q (for promotion)

For western chess variants algebraic notation counts ranks starting with 1, so in UCI we count ranks starting with 1. So there is no reason to change that for western variants.

In Xiangqi the notation either counts ranks from 1 with some implementations using 0 as abbreviation for 10, or starting with 0 going up to 9. And of course the rank-file order is different from western notation, the board orientation, the starting side, .... On the other hand the chess notation can be used with a1 being the left lower square for White (Red).

For cutechess a Xiangqi implementation has been suggested by @gaintpd https://github.com/cutechess/cutechess/pull/565 He tested his implementation using the engines maxqi, hoixiangqi, sjaakii, fairy-stockfish (largeboard). The former two engines do not support UCI, so they must have used CECP.

[cjssh1002]

In cute-janggi, uci mode is used. So cute-janggi can use ponder for janggi. janggi doesn't have a standardized protocol by default. Only FS first tried CECP with Xiangqi variant. The most important point here is that for the full functioning of FS, users want Ponder. So I want the coordinate problem to be solved well. There is no correct answer here. Engine developers and Winboard developers can define the janggi protocol. Because it is the first time. Ultimately, Chinese coordinates and chess coordinates are not Korean janggi coordinates.

[HGMuller]

I would recommend sticking to the UCI spec

This spec says nothing about how the ranks should be numbered. The problems with rank-file ordering, board orientation and starting color occur in Shogi, not in Xiangqi. The World Xiangqi Federation has defined standards for FEN and games, and the latter uses 0-9 for ranks, a0 being the lower left corner. There is a 'native' Xiangqi notation, but it is not a 2-d coordinate-based notation: it just numbers files, (in opposite directions for the players!) and uses + or - to indicate forward or backward relative motion.

As far as 'sticking' to things is concerned: there existed a prior implementation of UCI, introduced by Cyclone (a commercial Xiangi engine and GUI). Because the GUI became quite popular, many other engines have been using it as well, so that they could run under that GUI. So when I implemented support in WinBoard for UCI Xiangqi engines (through UCI2WB) I stuck to that standard, rather than creating my own incompatible new one. In fact the only reason that CECP uses 0-9 rank numbering for 10-rank board was to stay as compatible with UCI as possible. And this numbering does make sense; it avoids double digits, and does not preclude extension to even larger board, which the ugly kludge 10=0 would. I suppose this is why the WXF adopted it.

If CuteChess works with the current Fairy-Stockfish (or apparently Sjaak II) for Xiangqi in UCI mode, it will obviously not work with Cyclone (and several other Xiangqi engines). You might not care about that, but I am pretty sure the Chinese do.

In general it is a very bad thing to make the protocol dependent on the variant. If variants would all use their own unique notation in engine-GUI communication, general configurable multi-variant engines could no longer be configured to play that variant, as it would not just require alteration of the rules, but of the move notation as well.

What has happened in practice is that because of miscommunication the UCI protocol was unintentionally forked. We now have a UCI-C (Chinese version) and a UCI-W (Western version), which are incompatible for games on boards with 10 ranks.

[gbtami]

"In general it is a very bad thing to make the protocol dependent on the variant." This is why all UCI variant engines should starts rank counting from 1 for all variants :wink:

Can Cyclone GUI run UCCI xiangqi engines? If yes, I see no problem here.

[HGMuller]

The convention to use 0-9 counting in CECP is not dependent on the variant, but on the board size. To configure a multi-variant engine for a new variant, you would necessarily have to specify the board size, so it can be assumed it is always known. That you have to use variant-specific notation, such as the 6g6f in Shogi, can in no way be guessed from the rules.

I don't think the Cyclone GUI supports UCCI. I cannot be sure, as I don't have it. But the relevant question seems to be whether the Cyclone engine supports UCCI. The version I have doesn't.

[gbtami]

In UCI mode Fairy-Stockfish doesn't use variant specific notation (6g6f) for Shogi or for any other variant. This notation is used in USI mode only as I know.

[HGMuller]

I never said it did. I just pointed out that it would be bad to do so. No matter how common such a notation would be by Shogi players. Notation should be unambiguously derivable from the game rules, in a simple way. That the allowed range of rank numbers must be dependent on board size is obvious; in Chess you cannot allow a 9, but in boards with more than 9 ranks you cannot do without it. Whether it is better to extend the range at the upper or the lower end can depend on the situation.

But this is all idle talk. Fact is that a standard existed, and whether we like it or not, it was there, and widely used. And some people chose to not adopt it, and instead start according to their own, incompatible standard. So that we now have two mutually incompatible pools of GUIs and engines. I hope you are all happy with that, because I am not...

[alwey]

I do not see any necessity of shifting rank numbers down by one into UCI implementations for boards with exactly 10 ranks for western chess variants. It would be an ugly hack.

This kludge has been implemented in CECP, so let it rest there.

[HGMuller]

Wrong! This kludge was implemented in UCI. CECP just copied it from UCI, to not make it needlessly incompatible.

This attitude of starting new and incompatible protocols because of subjectively perceived cosmetic flaws ('ugly') is exactly what causes enormous amounts of hardship. Someone dislikes that the 'name' and 'value' keywords in the UCI 'option' command are redundant? Well, just leave them out and call the protocol UCCI instead. To make sure that there now will be two different pools of engines and GUIs that are not interoperable. Or force them to support both standards. Great thinking.

Making new and incompatible protocols should be considered a crime against humanity. They only create suffering, nothing else.

[alwey]

We are sticking to the UCI protocol definition, aren't we? It says the move format is long algebraic notation. An the notation for Grand chess starts from rank 1, no less.

[alwey]

This attitude of starting new and incompatible protocols because of subjectively perceived cosmetic flaws ('ugly') is exactly what causes enormous amounts of hardship. Someone dislikes that the 'name' and 'value' keywords in the UCI 'option' command are redundant? Well, just leave them out and call the protocol UCCI instead. To make sure that there now will be two different pools of engines and GUIs that are not interoperable. Or force them to support both standards. Great thinking.

Indeed UCI, USI and UCCI have some undesirable aspects. I am aware of many of the difficulties of bringing somewhat incompatible protocols into sync for match play and getting separated pools of engines to work with one another.

[gbtami]

"This kludge was implemented in UCI." No. This kludge was implemented in Cyclone. (What happened in UCCI or in CECP later is irrelevant here.)

Do you expect several GUIs and engines to be changed just because one commercial engine (are there any others doing this at all?) implemented this kludge in the past?

[HGMuller]

We are sticking to the UCI protocol definition, aren't we?

This is twisting words for your own purpose. 'Algebraic notation' just means concatenation of squares indicated by 2-dimensional cooridinates. Nothing more, nothing less. 6g6f also is a form of algebraic notation.

An the notation for Grand chess starts from rank 1, no less.

Yes, and the official WXF algebraic notation for Xiangqi (which has quite a bit larger number of players than Grand Chess) starts at 0. This is exactly the reason why protocol notations should not be made dependent on the particular notation habits for the variant.

No. This kludge was implemented in Cyclone. (What happened in UCCI or in CECP later is irrelevant here.)

UCCI and CECP indeed have nothing to do with this discussion at all. But Cyclone is not a UCCI engine. It is an UCI engine, and the first application of UCI to boards with more than 8 ranks. And it had wide following. As bad as the existence of UCCI and USI is, what you guys did now is even worse. For UCCI and USI you can at least see what is expected, because the initial command 'ucci' or 'usi' identifies the protocol. But you created a new and incompatible protocol without changing the name!

Note that I am not complaining that UCI does things differently from CECP. This issue is that there now are two different protocols that are not interoperable, but impossible to distinguish. How am I expected to support that in a GUI. That you don't care for running Chinese UCI engines in your GUI is your decision, but I would like to support both types.

Do you expect several GUIs and engines to be changed just because one commercial engine (there is any other doing this at all?) implemented this kludge in the past?

I could as well invert that question. Do you expect all Chinese engines and GUIs to be changed, just because Stockfish decided it knows better? Do you even know how to contact the authors of all these at all?

How many engines and GUIs are we actually talking about? CuteChess, PyChess, Fairy-Stockfish and Sjaak II? The authors of those are all easily reached, if not already here. That makes it certainly feasible. You could do it, if you wanted. Or, in other words, if it is not done, it is only because of your (collective) unwillingness.

As I see it, it is the fault of these authors that we are now in this mess, by not properly paying attention to "prior art". So it doesn't seem unreasonable to put the burden for cleaning it up on you too.

[gbtami]

"That you don't care for running Chinese UCI engines in your GUI is your decision, but I would like to support both types." As Fabian mentioned above Fairy supports UCCI for xiangqi. Isn't it usable with Winboard/Xboard?

"How many engines and GUIs are we actually talking about?" I could as well invert that question. How many Chinese engines and GUIs are using Cyclone style UCI-C protocol (starting rank numbering from 0)?

Btw you forget about http://www.xichess.com/ :)

[HGMuller]

As Fabian mentioned above Fairy supports UCCI for xiangqi. Isn't it usable with Winboard/Xboard?

I tried this, and it wasn't working 'out of the box'. AFAIK whether Stockfish plays Xiangqi or something else is determined by the UCI_Variants option, which is unknown in UCCI (and would have been expected to be named UCCI_Variants there had it existed). Even if Xiangqi would be automatically made default variant on reception of 'ucci', it would not solve the problem in Janggi or Grand Chess, which also have 10-rank boards.

Has SF's UCCI mode even been tested in a UCCI GUI (such as XQ Wizard)?

It seems plain silly to require Grand Chess to be played in UCCI mode. And it breaks the mechanism of engine-defined variants. For those the GUI gets to know the board size only after it selects the variant. But it has to decide on the protocol before that. So to guarantee proper operation it would always have to start in UCCI mode, just in case.

What you suggest is tantamount to stating "The UCI implementation is useless, always avoid it. Better stick to UCCI."

Note that none of this would help you to run Cyclone in PyChess. Cyclone cannot be switched to UCCI mode.

Btw you forget about http://www.xichess.com/ :)

First time I hear about that. But I don't see what it has to do with this issue. I even don't understand what it is. It seems a web-based GUI, but does it run UCI engines? And when I ask for an 'Analysis board' I getto see PVs in a coordinate system that does refer to rank 0.

[alwey]

xichess uses rank 0.

[HGMuller]

I could as well invert that question. How many Chinese engines and GUIs are using Cyclone style UCI-C protocol (starting rank numbering from 0)?

Oh, forgot this one. I am pretty sure there is more than one. As I wrote, the Cyclone GUI was rather popular, and many Chinese wanted their engine to be able to use it. There only has to be one other engine and one other GUI to top what we have here. And Cyclone was a top engine; failure to be able to run that would likely be reason enough for a Chinese to disqualify your GUI. The BingHe WuSi GUI might support UCI besides UCCI, I am not sure. Apart from that, why do you expect me to know the answer to that question? You can use Google just as easily as I can.

[gaintpd]

Dear all the friends, @gbtami @alwey @ianfab @HGMuller @cjssh1002 I am a native Chinese and the guy who adds xiangqi support for cutechess and the UCCI, UCI-cyclone protocol support for cutechess. I make some clarification.

The UCCI protocol was designed by the writer of eleeye(An ucci xiangqi engine) in the year of 2003(maybe 2005), which was inspired by UCI engine, and the start rank is from 0.The designer made some modification compared to UCI. The engine Cyclone @HGMuller mentioned is designed late than eleeye, and the designers for Cyclone has created a UCI clone protocol, we name it as UCI-cyclone. So the UCCI protocol is predate than UCI-cyclone.

The difference between UCI-cyclone and UCI is that the default variant for UCI-cyclone is xiangqi, and the default variant for UCI is standard chess(which was called international chess in China), another difference is that the rank numbering. The rank in UCI-cyclone starts from 0, while the UCI starts from 1. The difference is originated from the difference of the different board size of the two totally different variants.

This difference of the rank numbering makes the GUI writer and the engine writer have different viewpoint on the treatment of the variant whose rank number is 10. When I add the Musketeer variant in cutechess, I open an issue https://github.com/ianfab/Musketeer-Stockfish/issues/8 here, it has a hot discussion there.

When I finish the xiangqi support for cutechess, I start to add UCCI and UCI-cyclone protocol in cutechess, now these protocols are supported in my repo but was not upstreamed yet. The differences among the UCI, UCCI and UCI-cyclone is not that big. As a GUI writer, My viewpoint to this issue is that the GUI writer may need to try his/her best to add as many protocols as possible as the existed difference between the different engines.

[gaintpd]

Another fact may be out of awareness for the non-Chinese people is that the Cyclone GUI and xqwizard is rather old and stopped update. Now the most popular GUIs to play xiangqi in China now is Sharkchess, Binghe WuSi and the other two less popular ones. The XBoard is a great GUI that support xiangqi about 15 years ago, and now it is widely used by Linux users in China, another one is gmchess.

[ianfab]

Please, let's not make this discussion unnecessarily hard. It should be clear that there is not one single "right" solution and we unfortunately ended up with this situation of incompatible standards without anyone intending it. Neither did "we" intentionally decide to be incompatible with "UCI" Xiangqi engines, nor do "we" not care about conflicting standards.

Let us try to put together what happened:

• Xiangqi engine authors applied the UCI protocol to Xiangqi and in that way specialized it. To my knowledge that was not an extension of UCI (staying compatible with "chess UCI"), and the GUI at least needs to be told that it is "Xiangqi UCI" , i.e., that the engine plays Xiangqi, not chess.
• Several chess variant engine and GUI authors (at least the ones of Sjaak, cutechess, xboard/uci2wb, MV-SF, and Fairy-SF) started to use an extension of the UCI protocol using the UCI_Variant option to define the variant, and in this way staying compatible with pure "chess UCI" as long as the engine supports standard chess and uses that as the default variant.
• In the course of naturally generalizing this extended UCI protocol to arbitrary board sizes, most of those authors used the exact same counting of ranks for all variants, i.e., always counting from 1.
• Only then it became clear that this generalization conflicts with the specialized "Xiangqi UCI" definition.

So yes, it is very unfortunate that seemingly nobody was aware of the Xiangqi UCI convention when introducing the 1-10 numbering for Grand chess and the like (and only later for Xiangqi). Nevertheless, to me anything but using this very natural generalization would be quite awkward for multi-variant engines and GUIs. However, I of course do get the point that it is much easier to change on "our" side, since most relevant authors are still actively developing and are around here, while for the Xiangqi community it does not make any sense to switch, because who cares about other variants than Xiangqi?

Therefore, for me the main question is how the situation is regarding UCI vs. UCCI in the Xiangqi community. If UCI is just a mere remnant and new engines and GUIs mainly adopt UCCI, then it should be preferable to avoid the hack at the expense of a temporary incompatibility until UCI dies out in Xiangqi. If however UCI is still very relevant, we might have to swallow the pill and use the same convention for all 10 rank variants in order to avoid incompatibilities.

Edit: Thanks gaintpd for the clarification.

[gaintpd]

There is a flourish of engines in China that using the UCCI protocol as the specification and implementation code for UCCI is free and publicly available more than ten years ago, these engines can be found in the website of xqbase. But now the UCCI protocol gets less and less support. The protocols supported in the popular xiangqi GUIs in China as I mentioned above is UCI-cyclone as the play strengths for UCI-cyclone engines are much bigger than the UCCI ones.

Another thing I want to point out is that all the xiangqi GUIs and their users are the xiangqi users only and most of them are Chinese, while what we talked about here is in English. This is a parallel cosmic, the needs for the non-Chinese users should be satisfied firstly.

[gbtami]

@HGMuller I mentioned xichess.com because it uses Fairy's wasm port for analysis. xichess.com and pychess.org are not general purpose GUIs regarding engine support. Both are based on Fairy only.

Telling the truth supporting variant (or board size) dependent engine notations is a pain in ass in general. I was the happiest people when I noticed that I can throw off USI completely from pychess code base, after I noticed that UCI can be used for Shogi :)

[gaintpd]

@gbtami An unified protocol that can be applied for all variants is always a happy thing for both the GUI and engine Writers. But back to this problem, we have to face the divided reality that the existed differences between protocols, maybe we need to design a totally new protocol as a best solution.

[gbtami]

Please, don't! :) https://xkcd.com/927/

[alwey]

UCI-Cyclone is not really UCI compatible.

[ghost]

I think it's better to have a conversation that boosts the engine's performance intensity rather than discriminating between the right and the wrong. It seems that it will be able to maintain its current position only when the purpose is adjusted to the strength of engine performance. The current setting set by the developer is the most stable setting. If you know a better setting, I would like you to explain it with attached materials that can explain it(Example test results). The GUI is nothing more optimized than Winboard.

[gbtami]

Let me summarize this issue from pragmatic point of view. There are several competing chess engine protocols used for big board (> 8x8) variants over time: CECP, UCI, UCCI, UCI-Cyclone.

The suggestion here is to drop UCI support in favor of UCI-Cyclone.

Pros

Users of popular Chinese Windows GUIs supporting UCI-Cyclone only will be able to use Fairy-Stockfish. (After reading @gaintpd "parallel cosmic" post I think this use case is rather theoretic, say to least.)

Fairy will be compatible with Cyclone, so Winboard/Xboard users will be able to use it via UCI2WB instead of UCCI2WB.

Cons

Because this is a highly backward incompatible change, projects already using Fairy or already implemented UCI but not UCI-Cyclone will face to some pain and extra work, and Fairy will have an inferior implementation compared to what it has now. (Past big board variant games stored in pychess's database have to be converted also, grrrr).

[ianfab]

By the way, does a multi-variant UCI engine have any chance of knowing that it has to play xiangqi when used in a UCI-cyclone GUI? I guess UCI-cyclone does not include any such thing as UCI_Variant, so unless it is possible to set that option manually in the GUI, engines like Fairy-SF or Sjaak anyway can not work in there.

And if the only problem that would be solved here is to have UCI2WB compatible to both UCI and UCI-cyclone engines, then the adapter could simply add one more flag on top of the existing ones for usi, ucci, and 960 in order to explicitly enable UCI-cyclone (or to disable it, if you still want to have UCI-cyclone as the default). Since gaintpd anyway already did something similar in cutechess as far as I understand, that seems to be the least invasive change.

And if the need to have Fairy-Stockfish compatible to Xiangqi GUIs that do not support UCCI but only UCI-cyclone really comes up (e.g. because it improves a lot and becomes interesting for Chinese players), it can just do the same by allowing to explicitly enable UCI-cyclone at startup, which also gets rid of the problem of knowing the variant that needs to be played.

This of course would not get rid of the problem of having two standards, but it would simply admit that it already is the case and make it transparent to the user by explicitly being able to distinguish between them.

[alwey]

For cutechess I would suggest not dropping UCI because of UCI-Cyclone, but rather implementing UCI-Cyclone as a fifth protocol after UCI, CECP, USI, UCCI. The user has to chose the protocol for an engine configuration anyway.

If cutechess implemented auto-detection of protocols, UCI should be the default. Protocol detection /wrt UCI-Cyclone could be done by using one ore more test positions (startpos, FEN of chess startpos, FEN of xiangqi startpos) with the default variant and receiving one or two moves. If a Cyclonic engine gets startpos, it will make a xiangqi move and it will not like the chess FEN position.

By the way, does a multi-variant UCI engine have any chance of knowing that it has to play xiangqi when used in a UCI-cyclone GUI? I guess UCI-cyclone does not include any such thing as UCI_Variant, so unless it is possible to set that option manually in the GUI, engines like Fairy-SF or Sjaak anyway can not work in there.

The engine is in a bad position, because the communication is driven by the GUI. It will need some help.

And if the only problem that would be solved here is to have UCI2WB compatible to both UCI and UCI-cyclone engines, then the adapter could simply add one more flag on top of the existing ones for usi, ucci, and 960 in order to explicitly enable UCI-cyclone (or to disable it, if you still want to have UCI-cyclone as the default). Since gaintpd anyway already did something similar in cutechess as far as I understand, that seems to be the least invasive change.

AFAIK uci2wb uses an explicit option -c as indication to use UCI-Cyclone for variant xiangqi.

And if the need to have Fairy-Stockfish compatible to Xiangqi GUIs that do not support UCCI but only UCI-cyclone really comes up (e.g. because it improves a lot and becomes interesting for Chinese players), it can just do the same by allowing to explicitly enable UCI-cyclone at startup, which also gets rid of the problem of knowing the variant that needs to be played.

This seems to be a direct way to solve the problem for fairy. It gives fairy-stockfish the necessary help.

This of course would not get rid of the problem of having two standards, but it would simply admit that it already is the case and make it transparent to the user by explicitly being able to distinguish between them.

This is true, we have UCI as one standard and UCI-Cyclone as a derived standard.

[ianfab]

I did not really test it much, but https://github.com/ianfab/Fairy-Stockfish/commit/a4cadc76cb5c61a2ce26d1c31a42e499208c61b6 should be a simple solution to support enabling UCI-cyclone at startup in Fairy-Stockfish. Once enabled, Fairy-SF will assume that "UCI" means "UCI-cyclone".

[HGMuller]

UCI-Cyclone is not really UCI compatible.

That is correct. In UCI2WB I solved that by assuming UCI means UCI-Cyclone if the variant is Xiangqi. I would have to devise something more clever if there are now also Xiangqi engines that use orthodox UCI.

But for an engine the problem is not as big as for a GUI. IIRC the peculiarity of the Cyclone dialect is that it omits the 'position' keyword, and never uses 'startpos', but always send a 'fen'. So engines can be made to understand both dialects if they simply ignore the 'position' keyword, and interpret 'startpos' and 'fen' as if they are 'position startpos' and 'position fen', respectively. A GUI really has to be careful in whether it will send 'position' not. Getting it wrong usually chokes the engine.

Perhaps the UCI_Variant option can be used as the tell-tale sign of whether an engine would need orthodox UCI or UCI-Cyclone in Xiangqi. But then it is indeed the question how the GUI could ever get the idea that it could be switched to Xiangqi in the first place. This problem also exists for UCI variant engines that predate the introduction of the UCI_Variant option. (I have for instance a Komodo derivative that plays Seirawan Chess.) I guess a GUI would have to allow an engine without UCI_Variant option to be used in every variant.

But the burden to indicate this can also be put on the user, by implementing it as a separate protocol. The advantage of this is that when UCI-Cyclone is selected it can then be assumed the engine only plays Xiangqi. It doesn't help for other (unannounced) variants, though.

[ianfab]

Well, the information about the position keyword in my opinion is very relevant, because our whole discussion about having created an incompatible protocol is completely moot now, since there never was any UCI protocol for Xiangqi. It always was a separate protocol and GUIs need to be aware of that in order to decide which protocol to use (whether to send position or not). That makes the decision regarding ranks even easier, but of course renders my above implementation invalid because it won't accept omissions of position.

[alwey]

UCI-Cyclone is not really UCI compatible.

That is correct. In UCI2WB I solved that by assuming UCI means UCI-Cyclone if the variant is Xiangqi. I would have to devise something more clever if there are now also Xiangqi engines that use orthodox UCI.

I think in UCI2WB you indicated that explicitely using the -c option means UCI-Cyclone for xiangqi.

But for an engine the problem is not as big as for a GUI. IIRC the peculiarity of the Cyclone dialect is that it omits the 'position' keyword, and never uses 'startpos', but always send a 'fen'. So engines can be made to understand both dialects if they simply ignore the 'position' keyword, and interpret 'startpos' and 'fen' as if they are 'position startpos' and 'position fen', respectively. A GUI really has to be careful in whether it will send 'position' not. Getting it wrong usually chokes the engine.

Thank you for the hint, so the detection of UCI-Cyclone has to be done really carefully using explicit FENs. It is sad that engines block because the UCI spec demands engines should ignore everything that is not understood and carry on.

Perhaps the UCI_Variant option can be used as the tell-tale sign of whether an engine would need orthodox UCI or UCI-Cyclone in Xiangqi. But then it is indeed the question how the GUI could ever get the idea that it could be switched to Xiangqi in the first place. This problem also exists for UCI variant engines that predate the introduction of the UCI_Variant option. (I have for instance a Komodo derivative that plays Seirawan Chess.)

This old engine uses option name UCI_Seirawan type check default false - doesn't it?

I guess a GUI would have to allow an engine without UCI_Variant option to be used in every variant.

Cutechess only considers variants where a boolean engine option UCI_(Variantname) matches a known variant name. UCI_Chess960 is an example for a variant option. Selecting Capablanca Chess: setoption name UCI_Capablanca value true. BTW it was a good idea to use the keywordUCI_Variant for the combo option because no variant uses the name "Variant".

But the burden to indicate this can also be put on the user, by implementing it as a separate protocol. The advantage of this is that when UCI-Cyclone is selected it can then be assumed the engine only plays Xiangqi. It doesn't help for other (unannounced) variants, though.

[HGMuller]

I think in UCI2WB you indicated that explicitely using the -c option means UCI-Cyclone for xiangqi.

You are right! I remember that now. But I added that later, and remember feeling pretty clever to use the presence or absence of an option that merely confirmed the default as a signal to make some distiction. The question now is, why did the need arise to make that distinction? It can only be because I was exposed to a UCI Xiangqi engine that did require the 'position' keyword. Because that is the only effect the distinction causes. But I cannot remember which engine that was. And I would not have been able to run it under UCI2WB anyway if it had not counted ranks starting at 0.

This old engine uses option name UCI_Seirawan type check default false - doesn't it?

I don't recall that, but I don't have the engine on the computer where I am now.

[HGMuller]

Well, the information about the position keyword in my opinion is very relevant, because our whole discussion about having created an incompatible protocol is completely moot now, since there never was any UCI protocol for Xiangqi. It always was a separate protocol and GUIs need to be aware of that in order to decide which protocol to use (whether to send position or not). That makes the decision regarding ranks even easier, but of course renders my above implementation invalid because it won't accept omissions of position.

Be that as it may, a rigid attitude will not help to make things interoperable. Which I hope is our common goal here. If Fairy-Stockfish plays Xiangqi, it would be really a waste if it could not run on Chinese UCI GUIs, which all will be UCI-Cyclone. If it would have been a protocol with a different name, say UXI, you would probably not have had any reservation to add it as another setting for the protocol option, and accept that counting rank should start at 0. It would be just minutes work to do that, since it is so similar to UCI. And you did exactly that for UCCI, which must have been far more work. And for which we now hear is less important in China,

The problem is that it calls itself the same. So the engine cannot easily determine what is required of it. when it receives a 'uci' command. If we want move encoding to be different in the two dialects, we are perhaps lucky that the GUI will reveal itself when sending the position to the engine, which it must always do before GUI and engine can exchange any moves.

So the following solution suggests itself: If an engine in UCI mode receives 'fen' when it expected 'position', it interprets that as 'position fen', but also switches the protocol to UCI-Cyclone, which in practice only means that it now encodes ranks starting from 0. (Irrespective of variant or board size.) So in practice it just alters the start rank. If it receives 'position', it switches the start rank back to 1.

It seems to me that it would be an almost trivial change to Fairy-Stockfish to do this, as it must already have the code for altering the start rank, for the benefit of UCCI. And it would make Fairy-Stockfish 100% UCI-Cyclone compatible, (which must surely be worth the effort?), and still preserve the desired behavior in orthodox UCI.

Likewise, GUIs that support both UCI and UCCI must also already have code to alter rank numbering. So for those, supporting UCI-Cyclone just means putting it on the protocol menu, suppress sending of 'position' and avoid the use of 'startpos' when it gets selected. And assume the engine plays Xiangqi if it doesn't say otherwise.

Finally, I can let UCI2WB use Cyclone dialect for playing Janggi, to force rank numbering that is compatible with CECP, which would make me (and our Korean friends) happy.

I hope this will make everybody else happy to, and the computer-chess world a better place!

[alwey]

The question now is, why did the need arise to make that distinction? It can only be because I was exposed to a UCI Xiangqi engine that did require the 'position' keyword. Because that is the only effect the distinction causes. But I cannot remember which engine that was. And I would not have been able to run it under UCI2WB anyway if it had not counted ranks starting at 0.

Starting at 0 is not the problem, shifting ranks in UCI is the problem.

The World Xiangqi Federation has defined standards for FEN and games, and the latter uses 0-9 for ranks, a0 being the lower left corner.

So the notation itself uses rank 0. Then it is natural to use rank 0 in UCI for Xiangqi - this does not shift ranks. The Xiangqi board representation must deal with this.

The notation for Grand Chess starts from rank 1, UCI does not shift the ranks and the board representation deals with this.

Hexagonal chess uses ranks and files, uci does not shift them, the board representation deals with the coordinates.

[alwey]

Likewise, GUIs that support both UCI and UCCI must also already have code to alter rank numbering. So for those, supporting UCI-Cyclone just means putting it on the protocol menu, suppress sending of 'position' and avoid the use of 'startpos' when it gets selected. And assume the engine plays Xiangqi if it doesn't say otherwise.

Yes, UCI-Cyclone is a separate protocol in the UI and uses different keywords and uses Xiangqi as default.

Finally, I can let UCI2WB use Cyclone dialect for playing Janggi, to force rank numbering that is compatible with CECP, which would make me (and our Korean friends) happy.

You might also use the -c to force UCI-Cyclone for Janggi.

I hope this will make everybody else happy to, and the computer-chess world a better place!

I am not unhappy about leaving original UCI untouched.

[HGMuller]

You might also use the -c to force UCI-Cyclone for Janggi.

Indeed. But it would be much more user-friendly if this was automatic. Otherwise UCI Janggi engines would have to be registered with a startup command that explicitly launches UCI2WB, instead of just ticking the UCI checkbox.

It would even be nicer if UCI2WB would automatically use UCI-Cyclone in every case where the board has 10 ranks, so that the UCI and CECP coordinate notations match. Unfortunately this cannot be done, because UCI2WB is in general not aware of the board size. Only in cases where a variant is specified with board-size overrules, such as 5x5+5_shogi instead of 'minishogi', it would know the board size. (Which in that case is badly needed, as it is not possible to convert CECP to USI notation without knowing the board size.)

BTW, PGN games for Grand Chess saved by WinBoard do use rank counting 0-9.

[gbtami]

"BTW, PGN games for Grand Chess saved by WinBoard do use rank counting 0-9." This seems unnatural and rather unusual in grand chess scene. See all the materials available from https://en.wikipedia.org/wiki/Grand_chess

[cjssh1002]

HGMuller is a very friendly and grateful person. He is working on Korean janggi. Also, he is working on an openingbook for Korean janggi. However, I need some help from engine developers. We must help each other. I hope it will be a progressive debate.

[alwey]

BTW, PGN games for Grand Chess saved by WinBoard do use rank counting 0-9.

In my view this is a weakness, not a feature. In Grant Acedrex xboard uses ranks 1 to 12, which is correct. The CECP quirk shines through in the notation for games with 10 ranks.

[ianfab]

To be honest, I am not at all happy about the suggested solution (neither am I about being blamed to be uncooperative and stubborn, which is neither true nor does blaming contribute to a solution, but that is a different story...). Making UCI-Cyclone the default (or even the only supported one?) for games with 10 ranks encourages new engine developers to introduce a new protocol, because to my knowledge, for any game but Xiangqi (e.g., Janggi, Grand), there is not a single engine supporting UCI-cyclone, while there at least two using UCI. Or to put it with your words (exchange UCCI for yet another Xiangqi UCI dialect):

It seems plain silly to require Grand Chess to be played in UCCI mode

And for the user it should anyway not make any difference in which notation the moves are originally, since they should just be converted to CECP by the adapter, so using UCI-cyclone for Janggi would not have any advantage either.

If it would have been a protocol with a different name, say UXI, you would probably not have had any reservation to add it as another setting for the protocol option, and accept that counting rank should start at 0.

First of all, as you can see above I even tried to implement the protocol before you blamed me for having reservations against it. I do not have anything against supporting the protocol, I just am strongly opinionated against mixing it with UCI. If it were called UXI, the adapter would have a separate flag to enable it like for USI and UCCI, and it would not be the default over UCI depending on board size. That is why I emphasized that it is a separate protocol, just to clarify that it should be treated like one.

[HGMuller]

In my view this is a weakness, not a feature. In Grant Acedrex xboard uses ranks 1 to 12, which is correct. The CECP quirk shines through in the notation for games with 10 ranks.

This was intentional. It seemed universally better to use 0-9 counting on 10-rank boards. The WXF obviously thought so, and I felt the same. So I saw no reason to create an incompatibility between Xiangqi and other 10-rank variants. In particular in Grand Chess, where it would then put the Pawns on the second rank, just like on 8x8, so that it makes notation much more familiar. You can now also start with e4 or Nc3 there.

So your interpretation is in fact the reverse of what really happened. This is not a "CECP quirk". It existed as a notation standard that makes sense. UCI-Cyclone, UCCI and CECP just followed that standard, because there was no reason to deviate from it.

I admit that I first generalized it in XBoard by stipulating that on every board with more than 9 ranks counting would have to start at 0. This was a mistake, but at the time there were no variants with more than 10 ranks. Grant Acedrex was one of the first. Boards tend to be square or landscape (Xiangqi / Janggi are really exceptional in this respect), and 12x12 > 128, so most engines could not handle the larger variants. But once they could the inconvenience of the convention became clear, and after a discussion about it on TalkChess between me and some engine authors we decided to drop it for all boards other than with exactly 10 ranks. Hence Grant Acedrex now counts from 1.

[HGMuller]

Making UCI-Cyclone the default (or even the only supported one?) for games with 10 ranks ...

But this is not what I suggested as solution at all. What I suggested was to consider UCI-Cyclone a separate protocol, an using the omission of 'position' as the clue to recognize it. As you obviously wo't be able to deduce it from the initial 'uci', which both protocols have in common.

And to clear up a misunderstanding: I do not blame you for anything. I just pointed out that I did not expect that you would have any reservations against implementing a hypothetical UXI dialect, based on the fact that you did implement the less pressing and more troublesome case of UCCI. You obviously do have reservations against declaring 0-9 counting the standard for 10-rank boards in orthodox UCI, and I tried to respect that in the solution I proposed.

[cjssh1002]

The picture is a commercial janggi program in Korea. (name:janggidosa) This is independently developed by a single developer. Obviously, it can support English coordinates as well. It starts at 0. Although the protocol is clearly different, it can be set to express various coordinates. Just for reference.

[gbtami]

Just for reference also :)

Of course I know this has nothing to do with original issue (UCI rank numbering), but it DO has with how you save Grand games in .pgn files.

[HGMuller]

I am not aware of any organized competitions of Grand Chess, so people are likely to use whatever suits them considering the equipment they have. Where I live 10x10 boards are very common, but they never have coordinate inscriptions, as their reson of existence is International Draughts, which doesn't use algebraic notation, but simply numbers the squares 1 to 50. In other countries 10x10 boards are often hard to come by, and I don't know if the board in the image was a custom board made for Grand Chess, or taken from some other game.

Anyway, it seems to me Xiangqi tops Grand Chess in importance. And I definitely don't consider it a good thing for every variant to have its own unique notation conventions. IMO a format for an exchange medium had better be universal.

So for XBoard I adopted PGN as the single output format (although it understands a lot of other stuff on input). An official formal definition of PGN exists only for Chess, and the requirement it imposes that coordinates run from 1-8 and a-h, and pieces must be PNBRQK obviously has to be generalized to make it suitable for general chess variants. So the question arose how to generalize it.

As I consider creating new standards where existing standards would do the job pretty much a crime against humanity, I first looked around what already existed. And, lo and behold, it turned out that PGN was also in use for Xiangqi, and that the WXF had already defined an extension of PGN to boards as large as 9x10. And I doubt that any Grand-Chess game was ever saved as PGN, at that time. So I followed the existing extension of the PGN standard.

[gaintpd]

When I add support for UCI-Cyclone in cutechess, I treat it as a new protocol, and the c++ class name is definitely UxiEngine. In my personal opinion, Fairy-stockfish need not change one line of code on this issue, the GUIs should do the job. This is because fairy-stockfish is a multi-variant engine, not a xiangqi dedicated engine, it will break the consistence for other variant(s) whose rank number is 10, at last, I think seldom people out of China to use the UCI-cyclone engines in the non-Chinese world, the open source UCI-cyclone engines like Chameleon, Challenger, NewGG, whose source code are available in github, I bet there are very very little users in Europe, American and other non-Chinese area, not to speak of the commercial UCI-cyclone ones.

The designers of UCI-Cyclone protocol do nothing wrong to let the rank start from 0, but it is a big wrong to use the name of UCI. The best remedy is that the GUIs do some additional job as a workaround.