trishullab / PutnamBench

An evaluation benchmark for undergraduate competition math in Lean4, Isabelle, Coq, and natural language.
56 stars 8 forks source link

lean: check the populated solutions in CI #207

Closed eric-wieser closed 2 months ago

eric-wieser commented 2 months ago

The previous script hard-coded some paths, it now works them out from the location of the python script. This means it can now be run in CI, and the output checked for syntax errors, of which there were 5.

Most of these were due to errors in the python script, but a few were due to the syntax being broken by a lean upgrade.