search

SebLague / Chess-Challenge

Create your own tiny chess bot!
https://www.youtube.com/watch?v=Ne40a5LkK6A
MIT License
1.78k stars 1.07k forks source link

Final knockout tournament clarification #485

Open Algorhythm-sxv opened 9 months ago

Algorhythm-sxv commented 9 months ago

Hi, I noticed that the FAQ mentions that

Some percentage of the top bots will then be promoted to a second knock-out tournament

Is a knock-out format really appropriate? My concern is that a knock-out tournament cannot determine any placement except the top bot (i.e. you get no information about the second-best bot, since the second-best bot can be paired with the best bot and get knocked out early).

I feel that being able to definitively rank the top engines, as well as quantify strength differences between them would provide more interesting material for a video as well as more satisfying results for the competitors.

As one possible alternative to a knock-out tournament, TCEC (the premier computer chess competition) uses a multiple double-round-robin format, where each engine will play N openings from both sides against all other engines. The drawback of this particular format is that the number of games grows quite quickly with the number of competitors. An N-times double round robin with 32 engines would require nearly 2000*N games to complete. Depending on the number of cores available for the tournament this may not be feasible.

Other suggestions and discussions are welcome!

SebLague commented 9 months ago

I totally agree that round robin would be the ideal format, but I am concerned about how long it would take to run. Will make a final decision once I'm able to get a sense of the number of engines that are in contention for the top spot. I am open to hearing any suggestions though.

Algorhythm-sxv commented 9 months ago

I'm sure you would have people lining up to donate compute, and the chess programming community have some group servers that may help mitigate the bias from private hosting