search

Jlisowskyy / Checkmate-Chariot

Simple chess engine
MIT License
5 stars 0 forks source link

Quality and performance test framework creation #5

Open kryczkal opened 3 weeks ago

kryczkal commented 3 weeks ago

Performance todos:

kryczkal commented 3 weeks ago

Foundation for quality testing laid out in #4 and #10

kryczkal commented 2 weeks ago

Idea for testing framework:

  1. Run Checkmate-Chariot through a gauntlet tournament against many (eq. 10) engines of known and varying strength (ideally 1 engine for each 100elo above 1500, but then the tests would take too long).
    • UCI compliant engines will be taken from: CCRL - Index
    • the tournament will be run using: c-chess-cli
    • the output will be saved as .pgn
    • we will approve some sensible number for the number of games played against each engine
  2. the .pgn will be fed to Ordo | Bayeselo to retrieve an estimation of the elo
kryczkal commented 2 weeks ago

@Jlisowskyy

kryczkal commented 1 week ago

with #17 Uci implemented fully, we can now run c-chess-cli and get the elo of the engine

Jlisowskyy commented 1 week ago

We should decide about the time for each game, I belive it should be at least longer than bullet or blitz games. Do we would guarantee quality of the tests, there could be some shorter blitz like option for dynamic and fast tests also

Jlisowskyy commented 1 week ago

Also we will be running tests on release build so if the bug happens it would be nice to have some kind of script to reproduce the failure based on the command input sequence on the debug build

kryczkal commented 1 week ago

the Engine is currently below 1900 elo and i need to find worthy oponents