Consider what Game Phase means in Crazyhouse

ddugovic commented 8 years ago

Given that all Stockfish evaluations depend on game phase, it seems appropriate to define what game phase means in crazyhouse (so upstream changes are less likely to cause regression issues).

In normal chess, game phase is calculated based on material remaining on the board, so factors like king safety decrease and passed pawns increase as pieces exchange.

In crazyhouse, it's possible the opposite may be appropriate since exchanged pieces are placed back on the board with a vengenace!

@ianfab Thoughts?

Vinvin20 commented 8 years ago

Yes, there's no endgame in crazy house, you can drop that from Stockfish. Some ideas for Phases :

very early beginning -> no piece to drop, no king in danger
King(s) is under attack (checks have to be resolved if possible)
King(s) is very save (the engine has to find something to do others than king attack )

ianfab commented 8 years ago

I briefly thought about that when I implemented the rules, but I came to the conclusion that it probably is not that important for the time being, because the game phase (in the current definition) does not change as much in Crazyhouse as in other variants or standard chess.

However, of course, it makes sense to use this information. This again raises the unsolved problem of the definition of nonPawnMaterial for variants.

The problem is that the evaluation is sensitive to the game phase in terms like king safety, passed pawns, etc. While the king might be in more danger when we have more pieces to drop, passed pawns probably also are more dangerous when there are only few pieces on the board, since we cannot capture it by dropping a piece. So they are important in the same game phase in Crazyhouse (if my reasoning is correct), whereas in standard chess they are important in different game phases.

Edit: It might make sense to keep most of the things as is, but only change the king safety evaluation in the "endgame". Otherwise we would have to change a lot, and I do not know whether it would even yield an improvement.

ddugovic commented 8 years ago

@ianfab I agree game phase is less likely to vary; my main concern here is that upstream (official-stockfish) parameter tuning might break whatever changes we make.

I assume in terms of static evaluation, pieces in hand should count when considering game phase (and perhaps should count double!) such that exchanging pieces to "trade down to an endgame" is not a strategy to improve king safety, etc.

ianfab commented 8 years ago

Among others, I am currently running a test where I do not change existing king danger terms, but simply add values according to the number and type of pieces in hand. This could (over-)compensate for the gain in king safety by exchanging pieces. I think this could be a simple and effective measure. I also test a patch where I introduce general bonuses for pieces in hand to compensate for piece-square tables and mobility bonuses. By increasing the endgame bonus for pieces in hand, we could also achieve an effect on the decisions for or against exchanges. In both cases the values are just some small rather arbitrary values I came up with. If the ideas work, tuning session will follow.

Edit: The first idea already is an improvement with untuned values while the second one is about Elo neutral. Tuning will hopefully improve both.

sf-x commented 7 years ago

pieces in hand should count when considering game phase

This means the game phase is simply constant in the absence of promoted pieces.

such that exchanging pieces to "trade down to an endgame" is not a strategy to improve king safety

In normal chess Stockfish uses computed king safety for midgame and zero for endgame, you can simply use the same value for both "midgame" and "endgame" here , or perharps higher value for "endgame" (low material on board...)

ddugovic commented 7 years ago

Thanks, that makes sense.

It seems at this time there is a consensus that game phase should only refer to material on the board, and that crazyhouse evaluations should apply bonuses/penalties atop the standard chess implementation (and when that standard changes, bonuses/penalties need to be retuned).

ianfab commented 7 years ago

It perfectly makes sense to use the same king danger score calculation for midgame and endgame. The pieces in hand can then be accounted for by the ideas I mentioned above. The tuning session for these is still running.

Just to get an idea of the Elo sensitivity of changes to the midgame/endgame definition, I tested a version where the game phase was always set to PHASE_MIDGAME and the result was: ELO: 61.08 +-30.2 (95%) LOS: 100.0% Total: 500 W: 282 L: 195 D: 23

rpdelaney commented 7 years ago

In chess, the rook is valued higher than the bishop or knight because rooks tend to dominate the minor pieces in the endgame. But in the middlegame, especially in crazyhouse, rooks do not dominate the bishop or knight at all. Thus, stockfish over-estimates the value of the rook in crazyhouse.

ddugovic commented 7 years ago

@rpdelaney Why do you believe these automatically-generated and heavily tested values are incorrect? Can you provide reference games? http://github.com/ddugovic/Stockfish/blob/master/src/types.h#L280-L286

In my experience rooks are vital to providing mate (in combination with other pieces).

sf-x commented 7 years ago

Just to get an idea of the Elo sensitivity of changes to the midgame/endgame definition, I tested a version where the game phase was always set to PHASE_MIDGAME and the result was: ELO: 61.08 +-30.2 (95%) LOS: 100.0% Total: 500 W: 282 L: 195 D: 23

Did this experiment go anywhere?

ianfab commented 7 years ago

@sf-x This commit deals with the value and king danger of pieces in hand. Apart from that I do not know how my test result could be used to directly derive an idea for an improvement, except for simply tuning midgame and endgame values.

Vinvin20 commented 7 years ago

I read comments here : https://github.com/ddugovic/Stockfish/issues/112#issuecomment-258915586 It seems SF-zh doesn't develop its pieces fast enough in the opening.

Unihedro commented 7 years ago

Another game that is similar to crazyhouse in that it has piece dropping is probably shogi. Most shogi strategies do describe three phases:

Joban (序盤)

This is when you start and form your pieces for attack and defense. The aim of this phase is to position your pieces for better attack and stronger defense. You will also choose which opening strategy you will be using. For this, knowing Joseki (定跡=system opening moves) will help . Capturing and exchanging of the pieces will be minimal at this stage.

Chuuban (中盤)

This phase is when you initiate your attack moves and exchange pieces, hopefully to your advantage. You will try to converge where your opponent's King is. You judge the situation with the position of your pieces and also with the value of pieces you have on the board and on hand.

Syuuban (終盤)

This phase is when you will try to checkmate (詰み=Tsumi) or brinkmate (必至=Hissi) your opponent's King. The value of pieces at hand is not as important. Speed to which you close in your opponent is a top priority.

(source: http://www.shogishack.net/pages/shogi-strategies/)

These strategies are vastly different from regular chess and I think it's close to the essence of crazyhouse.

When I watch players playing bughouse, it is really entertaining because in addition to identifying the phases, they also have to actively balance their team power to lead to a win even if it's only a brinkmate. Unlike bughouse though, crazyhouse only has one board and yourself to count on, so an "endgame" is merely the fact that a decisive attack closing in a king is actively happening.

ddugovic commented 7 years ago

Syuuban (終盤)

This phase is when you will try to checkmate (詰み=Tsumi) or brinkmate (必至=Hissi) your opponent's King. The value of pieces at hand is not as important. Speed to which you close in your opponent is a top priority.

It seems to me this phase is characterized by either (or both) kings being unsafe and either (or both) players having pieces in hand, probably enough pieces to brinkmate. Thanks to the most recent commit there are per-player counts of ALL_PIECES in hand...

Vinvin20 commented 7 years ago

I had some idea about this subject :

Joban <- no such a phase in ZH. In Shogi that could take around 12 moves for each player to move pieces in their camp before starting the attack ( I found this comment in French on wiki : https://fr.wikipedia.org/wiki/Sh%C5%8Dgi#Les_d.C3.A9fenses ) but in ZH after 2 or 3 moves the players has already a pawn or a piece that could take something.
Chuuban : in ZH, all moves but the following phase (null moves are very interesting in Chuuban phase) ...
Syuuban : when there are open lines around the king (sac of piece(s) is interesting to open lines) or even sooner when there's some checks and when there are some pieces in hand to threat the king. Or "when there are more than 3 checks in a row, or instead of checks, "mate in 2" threats". It would be interesting to have piece value tuning for this phase !

sf-x commented 7 years ago

Given the amount of CPU time spent on crazyhouse tuning since this issue was opened, the current definition probably can't be changed without a long re-tuning effort. Is changing definition of the game phase more likely to give an improvement that most other changes? I don't think so.

ianfab commented 7 years ago

I agree. Many parameter values for crazyhouse already are quite different for "middlegame" and "endgame", so it's unlikely that changing game phase does not result in a regression. Furthermore, I think the current definition proved to be quite reasonable, so I see no reason to change it.

ddugovic commented 7 years ago

I agree that actually implementing this change would be a very large effort, and I'm not claiming that this is a high priority.

ianfab commented 5 years ago

I noticed that since 74131088 non-pawn material and game phase in crazyhouse should be more or less constant, if I understand correctly. I was surprised that reverting the changed definition seems to be a regression. Perhaps it requires some re-tuning to be able to revert it without a regression.

ddugovic commented 5 years ago

That sounds right; I wouldn't expect bench crazyhouse to change much, esp. for the start position.

Unsurprisingly, I found that the following position.h change causes bench crazyhouse to match again. But I wonder how non-pawn material per player could change the search, given that the total non-pawn material never changes:

 inline Value Position::non_pawn_material(Color c) const {
+#ifdef CRAZYHOUSE
+  if (is_house())
+      return  st->nonPawnMaterial[c]
+            + pieceCountInHand[c][QUEEN] * PieceValue[CHESS_VARIANT][MG][make_piece(c, QUEEN)]
+            + pieceCountInHand[c][ROOK] * PieceValue[CHESS_VARIANT][MG][make_piece(c, ROOK)]
+            + pieceCountInHand[c][BISHOP] * PieceValue[CHESS_VARIANT][MG][make_piece(c, BISHOP)]
+            + pieceCountInHand[c][KNIGHT] * PieceValue[CHESS_VARIANT][MG][make_piece(c, KNIGHT)];
+#endif
   return st->nonPawnMaterial[c];
 }

 inline Value Position::non_pawn_material() const {
+#ifdef CRAZYHOUSE
+  if (is_house())
+      return non_pawn_material(WHITE) + non_pawn_material(BLACK);
+#endif
   return st->nonPawnMaterial[WHITE] + st->nonPawnMaterial[BLACK];
 }

ddugovic commented 4 years ago

Having studied crazyhouse and shogi a bit more, I agree that the current game phase functionality works well.

ddugovic / Stockfish

Consider what Game Phase means in Crazyhouse #94

Joban (序盤)

Chuuban (中盤)

Syuuban (終盤)

Syuuban (終盤)