order of games in a tournament

[vondele]

When I run a multi-engine tournament like

./c-chess-cli -each tc=4+0.04 option.Hash=8 option.Threads=1 \
   -games 4000 -concurrency 250 -openings file=noob_3moves.epd order=random -repeat -resign 3 700 -draw 8 10 \
   -engine cmd=./stockfish.master name=r10e116 option.EvalFile=/users/vjoost/test/nnue-pytorch/run10/nn-epoch116.nnue \
   -engine cmd=./stockfish.master name=r0 option.EvalFile=/users/vjoost/test/nnue-pytorch/para03/run0/nn-last.nnue \
   -engine cmd=./stockfish.master name=r1 option.EvalFile=/users/vjoost/test/nnue-pytorch/para03/run1/nn-last.nnue \
   -engine cmd=./stockfish.master name=r2 option.EvalFile=/users/vjoost/test/nnue-pytorch/para03/run2/nn-last.nnue \
   -engine cmd=./stockfish.master name=r3 option.EvalFile=/users/vjoost/test/nnue-pytorch/para03/run3/nn-last.nnue \
   -pgn out-para02.pgn 2

it appears that all 4000 games are first played between the first pair of engines, next the 4000 games for the next, and so on.

I wonder if that is by choice, since it would be rather convenient for long running tournaments to have the engine pairs alternate. That allows for intermediate analysis of their ranking, and for the use of -games 100000 to basically run the tournament for as long as necessary till rankings are solid.

[lucasart]

Yes, this is the expected behavior. If you want to mix things up, you can use -rounds. The point is that using rounds=1 (default) minimizes the number of engine switch operations (which can be costly in ultra bullet testing).

Example with 3 engines (A, B, C) in a round-robin:

• games=1000: plays (A, B) x 1000 games, then (A, C) x 1000, then (B, C) x 1000. Only 2 engine switch operations here.
• games=100 rounds=10: 10 * ((A, B) * 100 + (A, C) * 100 + (B, C) * 100). Compromise here: 10x more engine switch operations (ie. 20, still not much for 3000 games); but you mix things up a bit more as you wanted.

Note that there is -gauntlet option, where the first engine playes the others. So you have (N-1) pairs, instead of N(N-1)/2 (eg. with 3 engines, (A,B) and (A,C) but not (B,C) which is in the round-robin but not the gauntlet).

This part is not really explained in the README. Perhaps I should write a section that properly explains tournaments.

[vondele]

OK, makes sense. Yes, I was expecting the performance as underlying reason. Since there is an option, I can easily adjust. Thanks for pointing that out.

BTW, the newest master works well also starting the engines with the 250 concurrency, so I need no changes to the sources to run.

I'll leave the issue open as a reminder for the README section, but feel free to close if you prefer.

[lucasart]

done