Evaluation.py failing on KeyError: 'test/0'

openai / human-eval

Code for the paper "Evaluating Large Language Models Trained on Code"

MIT License

2.31k stars 330 forks source link

Evaluation.py failing on KeyError: 'test/0' #10

Open briviere opened 2 years ago

briviere commented 2 years ago

I tried running:

evaluate_functional_correctness ./data/example_samples.jsonl

Getting the following error:

File "/Users/brianriviere/projects/human-eval/human_eval/evaluation.py", line 65, in evaluate_functional_correctness
    args = (problems[task_id], completion, timeout, completion_id[task_id])
KeyError: 'test/0'

Is there something I'm not doing correctly?

HenryHZY commented 1 year ago

What about "evaluate_functional_correctness data/example_samples.jsonl --problem_file=data/example_problem.jsonl"?

gench commented 1 year ago

EOFError

tanzelin430 commented 1 year ago

thanks Henry, your reply is really helpful