bkestelman
commented
1 year ago Another option is to skip representing moves entirely and just specify the state after a move. Some pros & cons:
Pros:
- the ML model might be simpler since it only needs to deal with state as input
- learning might be faster since the model won't get confused by situations where "the same move" played in different positions results in opposite results
Cons:
- learning might be slower because it might be harder to take advantage of situations where the same move makes sense in similar positions (this stands against the last pro; it's probably impossible to decide which argument is more important without testing both approaches)
- this is different from Alphazero's approach, so we would be going into uncharted territory (this could also be seen as a pro ;)
- this is different from how most reinforcement learning frameworks work, so we may have to adjust some of the standard algorithms
I'd like to try both approaches. The state-only approach actually sounds intuitively simpler to me, but if we want to start with the tried and true state-move pair that's fine too.
Need to choose a way to represent moves and check if they're legal.
Should take into account that we may frequently want to list all possible moves for a position (so we can assign probabilities to each move).
The Alphazero paper (p13) represents a move as the initial position of a piece, followed by a one-hot vector of 73 possible relative moves from the initial square (i.e. queen moves or knight moves). The other option is just to use the final absolute position instead of a relative movement. That would have the advantage of using only 64 values instead of 73 but may require slightly more work to check if the move is legal.