It started life as a copy-paste of //cc:concurrent_selfplay with some simplifications (it doesn't write training examples, reload models, soft pick, etc) and added support for playing two models on two separate trees.
I've done a cursory verification that the code is correct by playing a handful of models against each other and verifying that the winrates produced by //cc:concurrent_eval are close to those from //cc:eval. We should do some more testing before replacing the old eval code completely.
Once this code is pushed, we should do another pass over it to factor out logic shared between this and //cc:concurrent_selfplay.
I also dropped in a oneoff script that scrapes a directory of eval logs and generates a plot of their win rate.
This will eventually replace
//cc:eval.It started life as a copy-paste of
//cc:concurrent_selfplaywith some simplifications (it doesn't write training examples, reload models, soft pick, etc) and added support for playing two models on two separate trees.I've done a cursory verification that the code is correct by playing a handful of models against each other and verifying that the winrates produced by
//cc:concurrent_evalare close to those from//cc:eval. We should do some more testing before replacing the old eval code completely.Once this code is pushed, we should do another pass over it to factor out logic shared between this and
//cc:concurrent_selfplay.I also dropped in a oneoff script that scrapes a directory of eval logs and generates a plot of their win rate.