Closed mooskagh closed 4 years ago
Maybe I'm missing something but can't you use a single plane and set the squares on which the rooks have castling rights? In your example where the king is on e1 and the rooks are on a1 and b1, then the move to castling with the a1 rook would be e1a1 and with the b1 rook would be e1b1. If a1 on the castling plane is set, then a1 has castling rights; if a1 is clear, then no castling rights. Same for b1.
You could also split the single castling plane into a castling plane for white and a castling plane for black, if desired.
Because that is a convolution network, encoding castling rights with a single bit would be inconvenient for the NN. It will only "see" that flag while thinking about something close to that rook, and it will be hard to take castling rights into account when calculating remote parts of the board.
In theory propagating of the castling right is not a problem with SE-units. It's possible to make a constant filled plane from input plane with only a single position set using just one SE-unit. It's also possible to turn other planes on or off depending on the castling plane. Ordinary resnet would require multiple residual blocks to do the same.
Ok, then it's not much of a problem, so we don't need option 1. But is there any drawback of implementing it as option 3? (e.g. is there an argument to have just one square set to 1 instead of entire file(column)?)
RULES OF FISCHERANDOM CHESS
A little known and long-discarded offshoot of Classical Chess is the realm of so-
called "Randomized Chess" in its various forms.
Fischerandom Chess (hereinafter referred to as "F.R. Chess") stands for Bobby
Fischer's new and improved version of "Randomized Chess".
F.R. Chess uses algebraic notation exclusively.
At the start of every game of F.R. Chess, both players' Pawns are set up exactly as
they are at the start of every game of Classical Chess.
In F.R. Chess, just before the start of every game, both players' pieces on their
respective back rows receive an identical random shuffle using the Fischerandom Chess
Computerized Shuffler, which is programmed to set up the pieces in any combination, with
the provisos that one Rook has to be to the left and one Rook has to be to the right of the
King, and one Bishop has to be on a light-colored square and one Bishop has to be on a
dark-colored square. White and Black have identical positions. From behind their
respective Pawns the opponents' pieces are facing each other directly, symmetrically.
Thus for example, if the shuffler places White's back row pieces in the following position:
Ra1, Bb1, Kc1, Nd1, Be1, Nf1, Rg1, Qh1, it will place Black's back row Pieces in the
following position, Ra8, Bb8, Kc8, Nd8, Be8, Nf8, Rg8, Qh8, etc. (See diagram.)
Black
a b c d e f g h
┏━━━━━━━━━━━━━━━━━┓
8┃ ♜ ♝ ♚ ♞ ♝ ♞ ♜ ♛ ┃8
7┃ ♟ ♟ ♟ ♟ ♟ ♟ ♟ ♟ ┃7
6┃ · · · · · · · · ┃6
5┃ · · · · · · · · ┃5
4┃ · · · · · · · · ┃4
3┃ · · · · · · · · ┃3
2┃ ♙ ♙ ♙ ♙ ♙ ♙ ♙ ♙ ┃2
1┃ ♖ ♗ ♔ ♘ ♗ ♘ ♖ ♕ ┃1
┗━━━━━━━━━━━━━━━━━┛
a b c d e f g h
White
In F.R. Chess there are 960 possible starting positions, the Classical Chess
starting position and 959 other starting positions.
Of necessity, In F.R. Chess the castling rule is somewhat modified and broadened
to allow for the possibility of each player castling either on or into his or her left side or on
or into his or her right side of the board from all of these 960 starting positions.
Please note the following points:
a) In F.R. Chess, depending on the pre-castling position of the castling King and
Rook, the castling maneuver is performed by one of these four methods:
1. By on one turn making one move with the King, and making one move with the Rook
("double-move" castling).
2. By transposing the position of the King and the Rook ("transposition" castling).
3. By making only one move - with the King ("King-move-only" castling).
4. By making only one move - with the Rook ("Rook-move-only" castling).
Clarification: In F.R. Chess, whether you "double-move" castle, "transposition" castle,
"King-move-only" castle, or "Rook-move-only" castle, in all cases always both the King and
the Rook jointly are deemed to be castling, and to have castled.
b) In F.R. Chess when "double-move" castling, if the King is standing on the square
the Rook will occupy naturally you must move the King first and then the Rook; if the Rook
is standing on the square the King will occupy naturally you must move the Rook first and
then the King; otherwise whether you move the King first and then the Rook, or the Rook
first and then the King is strictly your choice.
c) In F.R. Chess, when "double-move" castling either the King jumps over the Rook
or else the Rook jumps over the King.
d) In F.R. Chess, when "King-move-only" castling the King always jumps over the
Rook.
e) In F.R. Chess, when "Rook-move-only" castling the Rook always jumps over the
King.
f) "a"-side castling (O-O-O), and "h"-side castling (O-O) are the F.R. Chess
equivalents of Queen-side or long castling (O-O-O), and King-side or short castling (O-O)
respectively, of Classical Chess.
g) In F.R. Chess:
1. White's "a"-side castling (O-O-O) is performed with the hitherto unmoved White King and
the hitherto unmoved Rook standing to its left. After White's "a"-side castling (O-O-O) the
White King stands on c1 and the aforementioned Rook stands on d1.
Black's "a"-side castling (... O-O-O) is performed with the hitherto unmoved Black King and
the hitherto unmoved Rook standing to its right. After Black's "a"-side castling (... O-O-O)
the Black King stands on c8 and the aforementioned Rook stands on d8.
2. White's "h"-side castling (O-O) is performed with the hitherto unmoved White King and
the hitherto unmoved Rook standing to its right. After White's "h"-side castling (O-O) the
White King stands on g1 and the aforementioned Rook stands on f1.
Black's "h"-side castling (... O-O) is performed with the hitherto unmoved Black King and
the hitherto unmoved Rook standing to its left. After Black's "h"-side castling (... O-O) the
Black King stands on g8 and the aforementioned Rook stands on f8.
h) When castling in F.R. Chess, you are not allowed to make any capture(s).
i) When castling in F.R. Chess, no other piece(s), yours or your opponent's, are
allowed to stand between the castling King and Rook.
j) In F.R. Chess, when in check, you are not allowed to castle. Or, put another way:
In F.R. Chess you are not allowed to castle "out" of check.
k) In F.R. Chess you are not allowed to castle into check.
l) When castling in F.R. Chess, your King is not allowed to move through check, or
shall we say any "checked" square (same thing), nor is your King allowed to jump over
your own Rook if and when said Rook stands on a "checked" square. (A "checked" square
is a square that is attacked by one or more of your opponent's pieces.)
m) In F.R. Chess, after completing your first move of "double-move" castling it is
quite possible and not at all illegal for your opponent to be in a position where, were it
your opponent's move, your opponent would be in stalemate, check, or checkmate.
Should after completing your first move of "double-move" castling one of these three
aforementioned situations exist you are to ignore it and make your prescribed second
move of "double-move" castling.
n) Exception: In F.R. Chess, when your first move of "double-move" castling, or at
least what could become your first move of "double-move" castling, also constitutes a
completely standard non-castling King or Rook move, the choice of whether to proceed
with your second move of "double-move" castling or to consider your move as an already
completed non-castling move is yours to make. You signify your choice for the former by
first making your second move of "double-move" castling and then pushing the
appropriate button on the chess clock, you signify your choice for the latter simply by
pushing the appropriate button on the chess clock.
o) Suggestion: When playing a friendly game of F.R. Chess without benefit of a
chess clock, to avoid any possible confusion or disputes, it might sometimes be a good
idea to announce to your opponent just before you castle, "I'm now going to castle."
p) In F.R. Chess:
1. Once the White King has moved, White is not allowed to castle for the rest of the game.
Once the Black King has moved, Black is not allowed to castle for the rest of the game.
2. Once the heretofore unmoved Rook standing to the left of the unmoved White King has
moved, White is not allowed to "a"-side castle (O-O-O) for the rest of the game.
Once the heretofore unmoved Rook standing to the right of the unmoved Black King has
moved, Black is not allowed to "a"-side castle (... O-O-O) for the rest of the game.
3. Once the heretofore unmoved Rook standing to the right of the unmoved White King
has moved, White is not allowed to "h"-side castle (O-O) for the rest of the game.
Once the heretofore unmoved Rook standing to the left of the unmoved Black King has
moved, Black is not allowed to "h"-side castle (... O-O) for the rest of the game.
4. Any appearances to the contrary notwithstanding, you are not considered to be castled
or to have castled unless you yourself have actually castled, i.e. performed the castling
maneuver. In other words: If you haven't castled, you haven't castled.
5. Once White has castled, White is not allowed to castle again for the rest of the game.
Once Black has castled, Black is not allowed to castle again for the rest of the game.
q) Tip: In F.R. Chess, when it is momentarily impossible to "a"-side castle (O-O-O)
because of the obstructing position of your other unmoved Rook, or vice versa, when it is
momentarily impossible to "h"-side castle (O-O) because of the obstructing position of your
other unmoved Rook, do not be confounded. These are frequent occurrences in F.R.
Chess. Play proceeds in accordance with the rules in this rulebook.
r) Remark: In F.R. Chess, depending upon the starting position, and how the game
develops, it may often occur that in the positions produced immediately after "a"-side
castling (O-O-O), one, two, or all three of the squares for White a1, b1, and e1 (for Black
a8, b8, and e8), and after "h"-side castling (O-O), one or both of the squares for White e1
and h1 (for Black e8 and h8) are not vacant, as they automatically are, for example, in
Classical Chess.
All the other rules are the same as in Classical Chess.
If you are ever in any doubt about the correct "before" and "after" castling positions
in F.R. Chess, please consult the following tables, showing all (84) possibilities of
castling, (42) for White, and (42) for Black:
All twenty-one White "Before" and the one "After" position of "a"-side castling (O-O-O):
Before: Kg1; Rf1, e1, d1, c1, b1 or a1. After: Kc1; Rd1.
Before: Kf1; Re1, d1, c1, b1, or a1. After: Kc1; Rd1.
Before: Ke1; Rd1, c1, b1, or a1. After: Kc1; Rd1.
Before: Kd1; Rc1, b1 or a1. After: Kc1; Rd1.
Before: Kc1; Rb1 or a1. After: Kc1; Rd1.
Before: Kb1; Ra1. After: Kc1; Rd1.
All twenty-one White "Before" and the one "After" position of "h"-side castling (O-O):
Before: Kb1; Rc1, d1, e1, f1, g1 or h1. After: Kg1; Rf1.
Before: Kc1; Rd1, e1, f1, g1 or h1. After: Kg1; Rf1.
Before: Kd1; Re1, f1, g1 or h1. After: Kg1; Rf1.
Before: Ke1; Rf1, g1 or h1. After: Kg1; Rf1.
Before: Kf1; Rg1 or h1. After: Kg1; Rf1.
Before: Kg1; Rh1. After: Kg1; Rf1.
All twenty-one Black "Before" and the one "After" position of "a"-side castling (... O-O-O):
Before: Kg8; Rf8, e8, d8, c8, b8 or a8. After: Kc8; Rd8.
Before: Kf8; Re8, d8, c8, b8 or a8. After: Kc8; Rd8.
Before: Ke8; Rd8, c8, b8 or a8. After: Kc8; Rd8.
Before: Kd8; Rc8, b8 or a8. After: Kc8; Rd8.
Before: Kc8; Rb8 or a8. After: Kc8; Rd8.
Before: Kb8; Ra8. After: Kc8; Rd8.
All twenty-one Black "Before" and the one "After" position of "h"-side castling (... O-O):
Before: Kb8; Rc8, d8, e8, f8, g8 or h8. After: Kg8; Rf8.
Before: Kc8; Rd8, e8, f8, g8 or h8. After: Kg8; Rf8.
Before: Kd8; Re8, f8, g8 or h8. After: Kg8; Rf8.
Before: Ke8; Rf8, g8 or h8. After: Kg8; Rf8.
Before: Kf8; Rg8 or h8. After: Kg8; Rf8.
Before: Kg8; Rh8. After: Kg8; Rf8.
Here is a PDF with Bobby Fischer's original RULES OF FISCHERANDOM CHESS (1996): RULES_OF_FISCHERANDOM_CHESS_by_Bobby_Fischer_1996.pdf
I will use Fischer's terminology from that document.
Hans-Walter Schmitt used the term "Chess960" in the Mainz Chess event, instead of FISCHERANDOM CHESS, and soon some other's followed.
Bobby Fischer said in an interview that he don't like the term "Chess960". He said they removed his name from it and branded it "Chess960" because some organisations don't want to be associated with his name after his live radio interviews. For those who like to respect Bobby Fischer I therefore suggest to not use the term "Chess960".
However, in PGN games for F.R. Chess games the "Variant"
tag can be "Fischerandom"
or "Chess960"
, so both should be supported.
See https://en.wikipedia.org/wiki/Chess960
To correctly record a Chess960 game in PGN, an additional "Variant"
tag (not "Variation"
tag, which has a different meaning) must be used to identify the rules; the rule named "Fischerandom"
is accepted by many chess programs as identifying Chess960, though "Chess960"
should be accepted as well. This means that in a PGN-recorded game, one of the PGN tags (after the initial seven tags) would look like this: [Variant "Fischerandom"]
.
Using Bobby Fischer's terminology from the original RULES OF FISCHERANDOM CHESS manual, we say "a"-side and "h"-side castling for Queen-side (O-O-O and... O-O-O) and King-side (O-O and ... O-O) castling, respectively.
The AlphaZero papers are here: https://arxiv.org/pdf/1712.01815.pdf https://deepmind.com/documents/260/alphazero_preprint.pdf http://science.sciencemag.org/content/sci/suppl/2018/12/05/362.6419.1140.DC1/aar6404-Silver-SM.pdf
In https://arxiv.org/pdf/1712.01815.pdf on page 13 the representation of the input features are explained. It looks similar in all AlphaZero papers as far as I can see.
AlphaZero Chess uses two planes for player P1 and two planes for player P2 to denote the possibility to castle Queen-side or King-side. As far as I understand it, these four planes are boolean values where the entire 8x8 planes are either filled with zero's or one's.
Likewise, the color plane indicating side to move works the same: The entire NxN size plane is filled with zero's or one's to indicate a boolean state.
I think the reason that the entire NxN plane is filled (and not just one bit or a row of bits) and set to the same vale (0 or 1), for the side to move (color) and castling rights, is that the neural network shall not associate this kind of state with a location (geographical location on the board).
Do you guys think that is correct? I don't know if this is the case. Maybe a neural network can figure it out even when not all squares are marked with the same boolean value, but it seems to me that by setting all NxN bits in a plane to the same value the neural network will not have to figure out if it is location based (and it is obviously not the case even for castling states, it is just a static state that is not affected by the location of any piece).
Therefore, if this is true, Lc0 should use the same representation here as AlphaZero Chess. We should make it as efficient as possible for the neural network to understand static states that are not related to geographical locations.
So the input features to Lc0 should then represent the right to castle as boolean states, four "flags" where the entire planes (four: two for each player) shall be set to zero's or one's. The names of the states can conveniently use Bobby Fischer's terminology: One 8x8 plane denoting White (or P1) "a"-side castling: All zero's if not possible. All one's if possible. One 8x8 plane denoting White (or P1) "h"-side castling: All zero's if not possible. All one's if possible. One 8x8 plane denoting Black (or P2) "a"-side castling: All zero's if not possible. All one's if possible. One 8x8 plane denoting Black (or P2) "h"-side castling: All zero's if not possible. All one's if possible.
Those are the only input states needed to the neural network when it comes to the castling stuff.
There is probably no need at all to have any input features to point out which "a"-side and "h"-side Rook's are the Rooks that the King can castle with. The neural network considers the position of all pieces and expresses through the policy head the prior probabilities for all actions (which can be moves, but can also be non-moves, like in Go one action is to "pass" and AlphaZero has a separate action output for "pass"). The actions includes moves that are illegal because of location of all pieces.
The way AlphaZero Chess expresses the action to castle Queen-side or King-side (the Classical Chess terminology) is simply King (on e1) to c1 (for Queen-side), or King to g1 (for King-side) castling. It has learned that when it expresses the "castle Queen-side" action the result is that the King ends up on c1 and Rook on d1, and the "castle King-side" action results in that the King ends up on g1 and Rook on f1.
That encoding "e1-c1" or "e1-g1" could have been anything I'm sure. It could be a separate action signal, just as in Go AlphaZero uses a separate action signal to express "pass".
Therefore the encoding can be anything to express castling "a"-side or castling "h"-side. Lc0 will learn how to set the probabilities for the policy head's output for castle "a"-side and castle "h"-side, respectively. It can thus be separate unique signals in the array of actions possible.
Then comes the other part that is not at all related to the input and output of the neural network. That is to encode which "a"-side Rook (if any) is the Rook that the King can castle with, and likewise which is the "h"-side Rook (if any) that the King can castle with.
That information needs to be stored with each position in the game tree and search tree, or at least the current analysis line of the search tree.
I can think of several representations. For example, an 8-bit byte, a bitmask, marking White's backrow with a '1' on the Rook or Rooks that are the Rooks the King can castle with. Since it is known where the King is, it is also known which Rook is the "a"-side and which Rook is the "h"-side, since it will be to the left side of the King and to the right side of the King.
With the "a"-side Rook placed on the "a"-file, the King on the "c"-file and the "h"-side Rook placed on the "g"-file, the bitmask would look like this: 10000010
White and Black have one such bitmask each. This representation needs 16-bits of storage per position. Not too bad. With this there is no need to have four separate flags to indicate the right to castle "a"-side and "h"-side for White and Black, because those four flags are stored in those 16-bits.
An alternative is to have four flags in a bitmask to indicate the possibility for "a"-side and "h"-side castling for White and Black, and then store the file location (a..h) as a 3-bit value 0..7 indicating which file the "a"-side Rook is placed on, and likewise a 3-bit value indicating which file the "h"-side Rook is placed on. If the flags indicating the right to castle "a"-side or "h"-side is set to false, then the 3-bit file location value is ignored (it can be any value).
This also requires 16-bits of storage for each position: 1+3 bits for White "a"-side castling, 1+3 bits for White "h"-side castling, 1+3 bits for Black "a"-side castling, 1+3 bits for Black "h"-side castling,
Maybe that is a better encoding.
NOTE: Castling is considered a King move.
For FEN notation, use the same as Stockfish & Co.
To express castling in the game/search tree: King-takes-own-Rook (the castling Rook) will work, since the program keeps track of which Rook it must be.
Ok, our training data format doesn't support generic planes, so training code would need changes too.
So it probably doesn't make sense to implement chess960 until we have protobuf-based training files. (Which won't probably happen before ~test70)
One part of the implementation will be to handle FEN that contains F.R. Chess positions. https://en.wikipedia.org/wiki/Forsyth–Edwards_Notation mentions how to handle F.R. Chess also.
Using the F.R. Chess games stored as PGN at CCRL http://ccrl.chessdom.com/ccrl/404FRC/ as examples:
[Event "CCRL 404FRC"]
[Site "CCRL"]
[Date "2019.01.13"]
[Round "202.1.313"]
[White "Houdini 6 64-bit"]
[Black "Stockfish 10 64-bit"]
[Result "1-0"]
[Opening "NBRKQRBN"]
[PlyCount "205"]
[SetUp "1"]
[FEN "nbrkqrbn/pppppppp/8/8/8/8/PPPPPPPP/NBRKQRBN w FCfc - 0 1"]
[WhiteElo "3660"]
[BlackElo "3730"]
FEN as defined for the Classical Chess starting position:
[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"]
would be defined like this in FEN supporting F.R. Chess:
[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w HAha - 0 1"]
The part of the FEN that express castling availability is different. In FEN for Classical Chess: 'K' means White can castle King-side or short castling (O-O), 'Q' means White can castle Queen-side or long castling (O-O-O), 'k' means Black can castle King-side or short castling (... O-O), 'q' means Black can castle Queen-side or long castling (... O-O-O),
In FEN that support F.R. Chess, castling availability is expressed by denoting the files (A..H or a..h) where the Rooks (that can be castled with) are positioned, in this order: 'A'..'H' if White's "h"-side castling (O-O) is possible with the Rook on the right-side of the King. 'A'..'H' if White's "a"-side castling (O-O-O) is possible with the Rook on the left-side of the King. 'a'..'h' if Black's "h"-side castling (... O-O) is possible with the Rook on the right-side of the King. 'a'..'h' if Black's "a"-side castling (... O-O-O) is possible with the Rook on the left-side of the King.
A few more examples:
[FEN "qnnbbrkr/pppppppp/8/8/8/8/PPPPPPPP/QNNBBRKR w HFhf - 0 1"]
[FEN "rnkrnbbq/pppppppp/8/8/8/8/PPPPPPPP/RNKRNBBQ w DAda - 0 1"]
[FEN "bbrknqrn/pppppppp/8/8/8/8/PPPPPPPP/BBRKNQRN w GCgc - 0 1"]
The neural network need to take into consideration if castling rights exist or not.
In My 60 Memorable Games, game 40 Fischer checks the opponents King and then moves the Knight back, so that the position of all pieces are the same, but Black has lost the right to castle (by moving his King). Fischer commented "Back where we started — but Black has lost the right to castle."
Therefore I think it is of paramount importance that the neural network has knowledge of the castling state.
The neural network need to see which Rook's can be castled with for both sides.
I didn't ask anything like that in Discord. :)
UCI protocol for Fischer Random Chess
• The engine has to tell the GUI that it is capable of playing Fischer Random Chess, and the GUI has to tell the engine that is should play according to the Fischer Random Chess rules. This is done by the special engine option UCI_Chess960. If the engine knows about Fischer Random Chess it should send the command 'option name UCI_Chess960 type check default false' to the GUI at program startup.
• Whenever a Fischer Random Chess game is played, the GUI should set this engine option to 'true'.
• Castling is different in Fischer Random Chess and the White King move when castling short is not always e1g1. A King move could both be the castling King move or just a normal King move. This is why castling moves are sent in the form King-takes-own Rook. Example: e1h1 for the White short castle (O-O) move in the normal Chess starting position.
• In EPD and FEN position strings specifying the castle rights with w and q is not enough as there could be more than one Rook on the right or left side of the King. This is why the castle rights are specified with the letter of the castling Rook's file. Upper case letters for White's and lower case letters for Black's castling rights. Example: The normal chess position would be: rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w AHah -
Do we close this issue @mooskagh ?
Indeed!
Let's have a tracking issue for Chess960 support.
The main question is how to encode castling moves as output from NN, and how to encode castling rights as input of NN.
For the output, the agreement seems to be that we should implement it as "king takes own rook" move, which actually we already do for normal chess. https://discordapp.com/channels/425419482568196106/445928688115122176/502374216180563968
For the input, we need to signalize somehow, which castlings are possible. Currently it's implemented as separate planes from king-side and queen-side castlings, which are filled with ones if side has right to castle.
For Chess960 this is not enough: Consider the following position: Rooks a1, b1. King e1. Queen-side castling rights are not lost. No pieces at c1, d1. Is castling possible in this position? The answer is "unknown", because it's not possible to distinguish between two situations:
So we need different encoding. Possible options:
I like option 3 the most. Please share your ideas/concerns.