Janggi ideosyncracies (XBoard compatibility)

[HGMuller]

Now that I have added 'janggi' as a standard variant to XBoard, the following points deserve attention:

• I am not sure that variants that only differ in determination of the game result deserve to be separate variants. In general I would consider an engine-defined option the natural way to modify the engine behavior in the required way. (But I understand that this can be troublesome in multi-variant engines.) In any case, the other Janggi flavors can now become engine-defined variants with parent jangqi, rather than xiangqi.
• The problem with spurious Pawn promotions (to nothing) is now solved, so the pieceToCharTable for Jianggi can now safely use the Pawn symbol again.
• Fairy-Stockfish highlights the King itself as destination when this King gets selected, thus inviting a second click on the King to be transmitted as a move like e1e1. It is not supposed to do this. As it happens the highlighting overrules the normal meaning of a second click on a selected piece, (which is deselecting it), but XBoard does not recognize this as a null move. In fact the resulting undefined behavior makes the King disappear. XBoard's user interface for entering a null move is clicking the inactive clock, and it is programmed to allow this (or not) on a per-variant basis. Which an engine can then exploit by selecting a variant as parent where it is allowed. Janggi might have to be a variant where it then is allowed.
• But I wonder where this notion that turn passing would be allowed comes from, as all rule descriptions I could find seem to clearly state that it is not...
• It could be argued that the kludge of double-clicking a King to indicate a turn pass is a better interface method than clicking the clock. Historically XBoard has always used the clock-click method to alter the side to move, though. (In Edit Position mode.) An advantage I see of the highlighting kludge is that the engine then can enable the turn pass depending on the position. But the downside is that it is a bit confusing that a second click on the same piece is no longer deselection. So if per-position enabling of null moves is desirable, I think it would be better to extend the CECP highlight protocol with an engine-GUI command to enable it. (Which the engine then could send spontaneously when its turn comes up, without waiting for any lift command.)
• How does Stockfish handle the rule that the initial location of Horses and Elephants can be selected by the players? This rule reminds me a lot of the Musketeer Chess 'prelude'. My attitude there was that this prelude in engine-engine play would always come from an opening line forced by the GUI, and that such opening lines might as well start with the FEN that resulted from the prelude. That pretty much seems to apply here too. For human-engine play I abused the ability of setup commands to show an arbitrary board position to the user, and the highlight protocol's ability to communicate mouse clicks to the engine, to have my Musketeer engine (KingSlayer-Aramis) conduct a graphical dialog with the user to select pieces. Something like that could be done here too, where the first setup command would show four ranks with white Horses and Elephants in different orders, and the user could click on one of those to select the setup he wanted. (Or, if black moves first, let the engine pick a random (or its preferred) setup for white, and then show that plus four ranks with black H&E to have the user select the black setup.) and then finally send the negociated start position in a final setup command for starting the game. An alternative is to just treat Janggi as a shuffle variant, and let the engine randomly decide which of the 16 valid start positions it will use in its setup command. The user can always hit 'New Game' untill he gets one that he thinks is acceptable. Since he would normally only have 4 options, this should not take too long.

[HGMuller]

A question: when the Bikjang rule is not used, would it be legal for Kings to face each other (as in Chess) or illegal (as in Xiangqi)?

[ianfab]

Thanks a lot for your feedback.

• I agree that introducing additional variants just for minor rule modifications is not nice, but since there was a strong request from Korean players to have them, I finally added them. One reason why I dislike using options in this specific case is that the modifications do not only impact adjudication, but even move generation. I yesterday pushed a commit to a branch https://github.com/ianfab/Fairy-Stockfish/commit/0775a2edb1d90bfa2029025b3c89c37d9c686109 to make Janggi a standard variant and use it as a parent variant for the other janggi rule sets.
• Based on your comment https://github.com/ianfab/Fairy-Stockfish/issues/180#issuecomment-705197516, does it mean the engine should not print the piece command for the pawn in order to avoid the issue? Or is there some other solution?
• I will revisit the handling of passing, it is clear it was a hack, but as far as I remember there were some issues I could not solve otherwise, but I will check it again and hopefully be able to consolidate it.
• Rule descriptions for Janggi that can be found on wikipedia or the like should be taken with a grain of salt, I had to learn that the hard way when I started to implement Janggi. One reason for that is also that the Janggi rules have changed over time and also slightly different rules are used in casual and tournament play, as well as in different regions (North/South Korea). However, after having got a lot of feedback from European and Korean amateur and pro players I am quite confident of the current rules used in Fairy-Stockfish (apart from the controversial repetition rules). Passing should definitely always be allowed (pseudo-legal), apart from situations where it is illegal due to check.
• Yes, if you have a good solution to improve the handling/input of passing, that is very welcome.
• I think of it in a similar way, and for now I just assumed that the setup of an alternative starting position is up to the user/GUI and Fairy-Stockfish does not know anything about it. I kind of like the idea of randomly shuffling on new game for Janggi, maybe I will do that.

Regarding bikjang, in Janggi it is never prohibited to cause the kings to face each other irrespective of the rule set. Whether bikjang is used or not then just decides if the game continues or might be adjudicated. However, when bikjang is used, it is illegal to make a move that is neither a pass (accepting the bikjang) nor breaks the bikjang situation, e.g., you can't just move the king forward or backward.

[gbtami]

Wasn't coding pass move as king moves in place was chosen because they just do the same OTB (turn king to other side)?

[HGMuller]

• It was an XBoard bug that 'piece' commands alter the promotion behavior; they are not supposed to do that. I now fixed this by having the Betza move generator also suppress last-rank Pawn promotion in Xiangqi/Janggi, just like the native generator. Redefining the Pawn move just tickled that bug, and is indeed something I usually don't do when I am satisfied with the existing Pawn move. (Which is one of the reasons I never noticed this bug, the other being that I never made any variants based on Xiangqi in the first place. For all other standard variants mandatory last-rank promotion is OK.) If you use the highlight protocol the only reason to redefine moves through 'piece' commands is to get better SAN disambiguation. But in this particular case that would not be an issue, because the Pawns start spaced one file apart, so there would never be any possibility for an ambiguous Pawn move even when XBoard imagines the Pawns do have sideway moves in places where they haven't. I noticed that XBoard doesn't automatically give the Pawns the desired behavior, because the test used for passing the River isn't resistand to odd-ranked board, leading to a asymmetric behavior. I should probably fix that too; then Crowded Xiangqi could be played without any piece or highlight commands at all.
• OK, if you feel the rules used are sufficiently reliably sourced, I will implement an always-allowed turn pass in XBoard. @gbtami - They might flip the King, but I suppose they also press the clock. I dislike having interface traits that are not homogeneous amongst variants, and in large Shogi variants you definitely cannot flip the King in OTB play, as that would promote it.
• I will give implementation of 'on-demand turn passing' some more thought, since Jianggi doesn't need it, so that there is no urgency.

[HGMuller]

And out of curiosity: how do the rule variations affect move generation? It seems a moot issue whether moves other than passing or breaking Bikjang are illegal or also valid acceptance of the draw.

[ianfab]

Since draws in Janggi usually are decided by material counting (at least in tournament play), it is relevant whether a player in his last move accepting bikjang can still capture a piece to change the material balance, or can only pass. Furthermore, when playing without bikjang, passing is never allowed when in check, but in a bikjang situation it is (see https://github.com/ianfab/Fairy-Stockfish/issues/94).

[HGMuller]

OK, I see. But I guess that the problem arises because it is really a misconception to view the acceptance of the Bikjang offering as a move. It is really just accepting the (implied) draw offer. In FIDE rules, when someone offers you a draw on a checking move, you can also accept the draw without the need to resolve the check first. That you signal the acceptance by flipping your King has no more significance than the habit to knock down your own King to indicate resignation.

[HGMuller]

I suppose it still will need some consideration that staying in Bikjang is illegal if you decline the offer. I guess I would implement this by recording the presence of the King connection as a separate bit flag from being in ordinary check in the (ply-indexed) stack I have to maintain anyway for detecting perpetual checking. In positions for which this flag is set, you can declare a win (King-capture score) if it was also set in the preceding ply (and otherwise ignore it). It is like a 1-ply-delayed check. If the rule doesn't apply, you just skip the test for the connection so that the flag gets never set. This assumes pseudo-legal move generation, though, which rejects the moves by testing for King-capture (including a pre-existing Bikjang) after the moves are made.

[HGMuller]

I ran into some unexpected trouble. Beware that this was with the old Fairy-Stockfish .exe I downloaded from here two days ago, which still plays janggi as engine-defined variant. (I never managed to compile Stockfish myself, and I am not sure if the latest changes are already available as Windows binary.)

I noticed that Fairy-Stockfish uses N for Horse and B for Elephant in FEN. Is this standard in Janggi? The XBoard implementation I made was using H and E, as in Xiangqi. So SF got quite upset when WinBoard sent it a FEN. (Which it accidentally did as a kludge to avoid having to send a null move.)

Another problem is that when I finally fixed XBoard to send the null move, SF did not accept it. From the debug log:

18290 >first : usermove @@@@ 18298 <first : Error (unkown command): @@@@

[ianfab]

Fairy-SF should not have any trouble with H or E in FENs for Xiangqi or Janggi. The background is that in Xiangqi in UCCI N/B are used, while CECP uses H/E, which is why Fairy-Stockfish internally uses N/B but allows H/E as aliases on input FENs in order to be compatible with CECP (it btw also uses these aliases for SAN). The same convention is also applied to Janggi. The issue might be that Fairy-SF confuses XBoard if it sends the setup and piece commands using the "wrong" symbols. This usually does not happen, since Fairy-SF skips sending those commands for standard XBoard variants (e.g. Xiangqi), but it of course does not yet for Janggi since it used to be an engine defined variant.

To my knowledge there is no standard, since I am not aware of any UCI, UCCI, or CECP engine for Janggi apart from Fairy-SF. The only other relatively well-known Janggi engine is Janggidosa, which however comes with its own GUI and does not expose any text protocol for engine communication.

The workaround that XBoard sends an FEN after a null move currently is handled explicitly in Fairy-SF by detecting whether the new FEN only differs to the current one by the side to move, and if so applies a pass move instead of setting up a new position. This circumvents the issue of not being able to undo a passing move, but from my perspective it would be preferable to directly send a pass move.

Yes, currently Fairy-SF uses "pass" and therefore does not understand "@@@@" as a passing move, I should change that. Oh, there also is a typo in "unknown", I should fix that as well :).

Edit: The current version from branch https://github.com/ianfab/Fairy-Stockfish/tree/xboard_janggi (Janggi as standard variant, @@@@ for pass move) is attached. fairy-stockfish-largeboard_x86-64.zip

[HGMuller]

After some experience trying to play Janggi with Fairy-Stockfish & XBoard/WinBoard, I can say the following things:

The CECP 'piece' commands are really a bad idea in Janggi. The moves they can specify are really too poor an approximation for the complex Janggi rules. (Palace confinement, diagonals, ban on CxC.) The moves indicated by the also supported highlight protocol would overrule them anyway as far as highlighting is concerned, and now that Janggi is a standard variant and XBoard has the exact moves built in, they do more har than good. In the standard variant itself XBoard can defend itself against an engine's 'piece' commands by switching legality testing on, but in engine-defined variants there is no defense, and 'piece' commands always overrule built-ins. And janggicasual etc. are still engine-defined. The piece commands there just wreck notation and checkmate detection.

So I would like to see an if(v->variantTemplate == "janggi") return; in ucioptions.cpp just before sending any piece commands. The stuff about the v->diagonalLines there can then all disappear, as I think only Janggi variants would ever use that. I cannot envision that anyone would ever want to use Janggi as a parent variant for anything other than Janggi, but if there would be demand for that, we would just have to suppress the piece commands for piece types normally present in Janggi.

Then there is the pieceToCharTable in the setup command, which serves two purposes: defining piece symbols and ID letters. In Xiangqi this never mattered much, as the setup command is completely ignored in a standard variant with legality checking on. (It is after all not legal to change the rules.) And there never is any reason to switch legality checking off in Xiangqi. So it doesn't matter whether the engine uses BN or EH, as long as in understands both on input FENs. In Janggi, however, there is the issue of the 4 rule variations, and 3 of those are played as engine-defined variant. And in engine-defined variants XBoard always takes the info in the setup command for gospel, as it is suppoed to have no clue about the rules itself. There really is no defense against that.

So it is very annoying if the engine changes the ID, altering move notation in 3 of the 4 sub-variants, compared to the standard Janggi. But even more disastrous is when it alters the symbols: this breaks the Polyglot opening book, as XBoard calculates the hash key by symbol rather than ID. (As the pieceToCharTable is also intended to adapt to local notations such as S = Springer etc.) So I would like the Janggi pieceToCharTable to be changed to "PH.R.AE..K.C.ph.r.ae..k.c." to make it possible for all 4 Janggi sub-variants to use the same opening book, and the same opening book that would work (and could be created) by XBoard with no engine loaded at all.

[ianfab]

I generally try to avoid hard-coding any variant-specific logic as much as possible, since that unavoidably causes more issues and work at some later point, but I think in this case there is a relatively safe and isolated workaround by returning a hard-coded setup command in case all of parent variant, starting FEN, and piece types are matching to Janggi like in https://github.com/ianfab/Fairy-Stockfish/commit/ca17aa027c1bdedec016a2482af060fbc0e6ab56. This should be very unlikely to interfere with any user-defined variants, so I would be fine with this change if it is sufficient from your point of view. That would also allow to remove Janggi from the "standard variants" so that the setup command is still sent, which avoids completely breaking backwards compatibility with XBoard versions that do not have Janggi as a build-in variant.

[HGMuller]

I agree it is a hard problem to do this in a clean way. One could suppress all 'piece' commands of pieces that normally occur in a parent variant, and have the same move. This supposes that the engine has some way to compare move descriptions, and that it implements every variant that is used as a parent variant. (Which is true for the Janggi group, but need not be generally true.)

Janggi / Xiangqi is a bit of a special case, because the built-in moves there are of a type that cannot be conveyed by the piece commands. It is this non-representability (e.g. due to confinement to zones) that causes the problem; if the piece command would just send the move that was already built in, there is no harm in that. (There was for the Pawn in Xiangqi, but that was unintended and qualified as a (now fixed) XBoard bug.) The only pieces with unrepresentable built-in moves are E and A in Xiangqi, and P, A, C, R and K in Janggi. Perhaps we should declare these piece symbols 'sacred' in these parent variants, and never send any piece commands for them. If someone wants to define a variant that uses Xiangqi/Janggi as parent, but which contain pieces that move different from the pieces originally present, there are plenty of other symbols to choose from.

I don't see any possibility to stay compatible with XBoard versions that do not support Janggi: to do that you would have to specify Xiangqi as a parent.

[ianfab]

I commited the change to skip piece commands and change the setup command for Janggi variants to master. I will perhaps also remove the highlight of passing moves, but I have a question regarding that. @HGMuller In edit and analysis mode one can play a passing move by clicking the clock, but when playing that tries to claim a win on time instead and fails. So how does passing work when playing?

[cjssh1002]

Yes you are right, this is buggy. If you use pass in analysis mode, the turn does not change. There is no problem in human VS AI. I click the clock to "pass".

[HGMuller]

This was an XBoard bug in the initial Janggi implementation. For clock clicking to work as turn pass in playing mode the current (parent) variant must be indicated as allowing null moves, and I had not made that indication for Janggi yet. A problem here could be that you are using XBoard from my on-line git repository, which might not have all patches in them that I applied directly to the source files I have on Windows, to make new WinBoard-AA packages for cjssh1002 to test. And there is a lot of patches (in connection with book creation / probing) that don't work yet.

But I will check and make sure the 'enable null move' patch is committed to my on-line repository.

[Edit] Well, it seems the patch is present there, as forelast commit in the v4.9.x branch.

[ianfab]

Oh, thanks, seems like I had used the version from two commits earlier in testing, now it works.

So from my perspective the only point that is still open here is to potentially implement a way to alter the starting position either via shuffling or a user dialogue.

[HGMuller]

When this issue came up in Musketeer Chess, I made a slight enhancement of the 'setup' command to conduct such dialogs by graphical means. An alternative would be to conduct it as text, through the CECP 'askuser' command. But that would still pose a problem, namely that in an engine-defined variant XBoard expects a 'setup' reply to 'variant', and before the dialog is completed the engine doesn't know what initial position to specify. I suppose it would be possible for the engine to use 'askuser' and wait for the reply before it answers the 'ping' command XBoard uses to test if a 'setup' reply can still be expected.

Anyway, I made it possible for engines to indicate a 'temporary setup' by prefixing the parent-variant name with an exclamation point. Such a temporary setup will be accepted as a reply to the 'variant' command, but it will not make XBoard ignore any further 'setup' commands, until a final one (without '!') is received. This allowed the engine to show arbitrary board positions to the user, while the highlight protocol informs it what the user clicks on those through 'lift' commands. By replying to the 'lift' commands with a next 'setup', it is prevented the user can ever enter a complete move in this stage. I think this method is highly superior to any text-based dialog, in terms of user experience.

The way it would work in Janggi is that after the variant Fairy-Stockfish would sent a 'setup' with a board that shows the 4 possible white back-rank constellations, as it is set for playing black at that stage, and expects white to choose first. It will then either receive a 'lift' command due to the user picking one, or a 'go' command to play white. (And it will be knowing whether it is playing against another engine, through the CECP 'computer' command.) If a 'lift' command indicates the white choice, the engine can make its own choice for black, (a small internal book for it seems the best way to handle that), send a final 'setup' with the negociated start position, and wait for the user to move. If, OTOH, it receives 'go', it can send a temporary setup that shows the user the engine's chosen white constellation, offering a choice between 4 black back-rank constellations, and start waiting for 'lift' to know which one the user picked. It then replies to that with the final 'setup', and starts thinking.

When playing against another engine it does nothing of this; it just sends the final 'setup' with the standard start position, or a randomly picked one. The GUI willl likely overrule it with a 'setboard' command from a book of opening lines, as engine-engine games are typically not played from the start position, to force game diversity.