search

openai / evals

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Other
14.76k stars 2.58k forks source link

Allow for evals with no args #1517

Closed thesofakillers closed 6 months ago

thesofakillers commented 6 months ago

As raised in #1515, the args field of EvalSpec is optional. Therefore it is possible for evals with no args to exist. Here args is None.

However, currently our arg overriding code mistakingly does not support this API, since it assumes args is not None.

This PR addresses the issue with an if statement. Fixes #1515