search

FudanSELab / ClassEval

Benchmark ClassEval for class-level code generation.
MIT License
108 stars 5 forks source link

Evaluation script fails with error #5

Closed ramsey-coding closed 2 months ago

ramsey-coding commented 4 months ago

Evaluation script fails with the following error:

python evaluation.py --source_file_name model_output --eval_data ClassEval_data --greedy 1

Error writing XML: [Errno 2] No such file or directory: ''
Error extracting files: [Errno 2] No such file or directory: ''
Error extracting file: [Errno 2] No such file or directory: ''
ramsey-coding commented 4 months ago

@mingwei-liu can you point me out how to run the evaluation framework?

code-rex1 commented 4 months ago

@mingwei-liu @Humbertzhang @csguojin @skyleaworlder I am also having the same problem. At first I ran the inference and generated the model_output.json.

Then triggered the evaluation script with:

python evaluation.py --source_file_name model_output --eval_data ClassEval_data --greedy 1

However, I am also having the same problem. Can any one please help me out?

wkx228 commented 4 months ago

Perhaps you can check if there are any generated result files in the folder output/result such as pass_at_k_result.json. If so, these error messages may be output during the execution of the test cases, and do not affect the evaluation process. Otherwise, please check if the files ClassEval_data.json and model_output.json are placed in the correct folder.