search

kiudee / chess-tuning-tools

A collection of scripts aimed at efficiently tuning chess engine parameters.
https://chess-tuning-tools.readthedocs.io/en/latest/
Other
52 stars 13 forks source link

Support matches against multiple engines #39

Open kiudee opened 4 years ago

kiudee commented 4 years ago

Being able to tune against multiple engines would be a nice to have feature, but would require considerable changes to the current script:

Claes1981 commented 3 years ago

In case of 2 games per iteration I think the opponent engine could be chosen at random, alternatively by alternating in a fixed sequence. Maybe those approaches could be used in case of more games per iteration as well. Not sure if the number of game pairs per iteration must be divisible by the number of opponents, or if the in other cases possible uneven results, could be treated as noise.

Claes1981 commented 3 years ago

I have now implemented the possibility to tune against multiple engines in my fork (commit https://github.com/Claes1981/chess-tuning-tools/commit/6b0fdf1b7fb45f0c33490bd92fd1dcf8ace978eb and commit https://github.com/Claes1981/chess-tuning-tools/commit/6f767c3f572c2b3089a47968d821150ed4c43b0e, there might still be bugs, though). It chooses "engine2" at random at each iteration. In case of 2 games per iteration, that means it might switch engine every 2 games. In case of more games per iteration, it means that it keeps the same engines through all games of that iteration, which I think is not ideal. Ideally it should choose an opponent at random every two games, I think.

kiudee commented 3 years ago

That looks like a straightforward way to implement a simple random selection. I have used such a strategy in the past. It is true that it will be slightly confusing to the optimizer, if the optimization landscapes are flipping back and forth randomly.

If you want you can cherry pick those commits onto a new branch and submit a pull request for review.

Claes1981 commented 3 years ago

If you want you can cherry pick those commits onto a new branch and submit a pull request for review.

Not sure I know how to do that, but I might try.

You are of course welcome to copy some or all of the code, or the ideas, into Chess Tuning Tools, if you want. :)