trishullab / PutnamBench

An evaluation benchmark for undergraduate competition math in Lean4, Isabelle, Coq, and natural language.
45 stars 5 forks source link

lean: Disable autoImplicits #202

Closed eric-wieser closed 3 weeks ago

eric-wieser commented 1 month ago

These are a common source of misformalizations. For instance,

https://github.com/trishullab/PutnamBench/blob/a945338fdc523d7612108a1d7fc07602b19cdde4/lean4/src/putnam_2017_b2.lean#L14

has the parenthesization around the summation incorrect, and so i is not in scope. As another example,

https://github.com/trishullab/PutnamBench/blob/a945338fdc523d7612108a1d7fc07602b19cdde4/lean4/src/putnam_1978_a3.lean#L8-L10

is misformalized as "for any Polynomial with name X", not for "the monomial Polynomial.X". With this option on, this now gives an error that X doesn't exist.

The solution is either to add the missing arguments (when the formalization is correct), or fix the formalization (when it is not).