Scoring bug

[hauensteina]

Dave Jarvis recently mailed me the attached screenshot and sgf. Katago thinks a large group with two clear eyes is dead. Happens on all KataGo versions and networks that I have, both from Mar 2024 and several years old.

scoring_bug_sgf.txt

[lightvector]

Thanks for the report! I don't think there is a bug in KataGo.

I'm guessing it's a bug in the GUI for failing to detect that the SGF contains illegal moves, or for ignoring the error message that KataGo reports when it rejects the illegal move. The first illegal move in the SGF appears to be on move 218 - this is an illegal ko capture by black. The very next move 219 also makes no sense, white plays a stone on the same spot that black played, which would be illegal even if you assumed that the black's ko capture was legal.

Right now, I think the GUI is probably incorrectly ignoring when KataGo rejects the illegal move, and proceeding as if the move were legal anyways. This results in the GUI showing a board state that is inconsistent with the board state that KataGo is providing the scoring or analysis for.

For example, if the GUI is using GTP, then as documented in the GTP spec at https://www.lysator.liu.se/~gunnar/gtp/gtp2-spec-draft2/gtp2-spec.html#SECTION00071000000000000000 at 6.3.3:

* play
arguments   move - Color and vertex of the move
effects         A stone of the requested color is played at the requested vertex. The number of captured stones is updated if needed and the move is added to the move history.
output          none
fails           syntax error, illegal move. In the latter case, fails with the error message ``illegal move''.
comments        Consecutive moves of the same color are not considered illegal from the protocol point of view.

So the GUI can parse the response to the play command to detect when "illegal move" is reported, and alert the user of the illegal move or react in some reasonable way, such as rejecting the move as well, instead of displaying the wrong analysis.

[hauensteina]

Indeed. And all these years not a single person complained that I don't flag illegal ko captures. Unbelievable. Will be fixed soon.

Thanks,

Andreas

On Thu, Mar 14, 2024 at 1:50 PM David J Wu @.***> wrote:

Thanks for the report! I don't think there is a bug in KataGo.

I'm guessing it's a bug in the GUI for failing to detect that the SGF contains illegal moves, or for ignoring the error message that KataGo reports when it rejects the illegal move. The first illegal move in the SGF appears to be on move 218 - this is an illegal ko capture by black. The very next move 219 also makes no sense, white plays a stone on the same spot that black played, which would be illegal even if you assumed that the black's ko capture was legal.

Right now, I think the GUI is probably incorrectly ignoring when KataGo rejects the illegal move, and proceeding as if the move were legal anyways. This results in the GUI showing a board state that is inconsistent with the board state that KataGo is providing the scoring or analysis for.

For example, if the GUI is using GTP, then as documented in the GTP spec at https://www.lysator.liu.se/~gunnar/gtp/gtp2-spec-draft2/gtp2-spec.html#SECTION00071000000000000000 at 6.3.3:

• play arguments move - Color and vertex of the move effects A stone of the requested color is played at the requested vertex. The number of captured stones is updated if needed and the move is added to the move history. output none fails syntax error, illegal move. In the latter case, fails with the error message ``illegal move''. comments Consecutive moves of the same color are not considered illegal from the protocol point of view.

So the GUI can parse the response to the play command to detect when "illegal move" is reported, and alert the user of the illegal move or react in some reasonable way, such as rejecting the move as well, instead of displaying the wrong analysis.

— Reply to this email directly, view it on GitHub https://github.com/lightvector/KataGo/issues/911#issuecomment-1998461617, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH54SBSZJ73D5G2EHRCNDTYYIEQJAVCNFSM6AAAAABEWP4XMGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJYGQ3DCNRRG4 . You are receiving this because you authored the thread.Message ID: @.***>