reidmcy
commented
2 years ago Something like this is something we've internally talked about a lot, building cheat detection systems for chess, but we don't have any datasets to use for training so publishing a paper would be tricky. If you know of any datasets for chess cheating I'm all ears.
When I have tried adding extra information to the models the improvements in prediction accuracy have been minimal, but there are some newer model architectures that might do better than the Leela style ResNet I tested with.
There are two other papers by me that would be even better for detecting cheating: Individualized Maias and the Chess Behavioral Stylometry papers. These have been focused on medium skill players, not high level play but the methods should generalize. You can take a sample (larger or small respectively) of games and compare it to a set of other games to see if the player is making playing with unusual moves/style.
P.S. @asdfjkl I enjoyed your book
asdfjkl
codeisnotfunjk
With all that cheating debate going on....
I am wondering if Maia or a Maia-like approach could be used to provide additional input to a cheat detection system?
After all, you reported in your papers that with Maia you get much higher human move predictions (depending on rating class) than with say stockfish.
Suppose we take not only the position & rating, but also extra-information such the remaining time of the player into account. Could a network be trained in such a way that the, say top3 move predictions amount to an (80%/90%/almost 100%) move prediction for a given position?
Given a game, this would then provide at least guidance to those positions where a human player (a potential cheater) played moves that the network classifies as "non-human".
Instead of a predictor one could of course also try to create a distiguisher network that given a position and a move tries to predict with a certain amount of probability if that was an engine or computer move...
There might be other creative ways... This might be complete nonsense and not work in practice, but given that you trained Maia and have a much better intuition on what a trained network is capable of, I would like to hear your opinion...