Closed jswang closed 6 months ago
When running the example provided in https://github.com/openai/evals/blob/main/docs/run-evals.md: oaieval gpt-3.5-turbo test-match, the following error is encountered out of the box:
oaieval gpt-3.5-turbo test-match
File "/Users/juliewang/Documents/evals/evals/cli/oaieval.py", line 167, in run eval_spec.args.update(extra_eval_params) ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'update'
This is due to this line: https://github.com/openai/evals/blob/main/evals/cli/oaieval.py#L158. The args field of EvalSpec is optional: https://github.com/openai/evals/blob/0dc0ba455194f5c9c4566ce415a2b2714f56e2db/evals/base.py#L58, and in the test-match case, eval_spec.args is None.
args
EvalSpec
test-match
eval_spec.args
None
No response
macOS
Python 3.12.2
openai-evals commit b5853eb4fed5001188671e87b20d8f8261b57e63
Thx for raising this! I've opened a PR that should address it.
Describe the bug
When running the example provided in https://github.com/openai/evals/blob/main/docs/run-evals.md:
oaieval gpt-3.5-turbo test-match
, the following error is encountered out of the box:This is due to this line: https://github.com/openai/evals/blob/main/evals/cli/oaieval.py#L158. The
args
field ofEvalSpec
is optional: https://github.com/openai/evals/blob/0dc0ba455194f5c9c4566ce415a2b2714f56e2db/evals/base.py#L58, and in thetest-match
case,eval_spec.args
isNone
.To Reproduce
oaieval gpt-3.5-turbo test-match
Code snippets
No response
OS
macOS
Python version
Python 3.12.2
Library version
openai-evals commit b5853eb4fed5001188671e87b20d8f8261b57e63