bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
710 stars 183 forks source link

When I evaluated the dataset APPS, I got the error RuntimeError: stack.size() >= frames.back().function->n_inputs INTERNAL ASSERT FAILED #108

Closed Luowaterbi closed 1 year ago

Luowaterbi commented 1 year ago

I've tried many nodes and this error is reported. According to this link it seems that the torch version needs to be upgraded, but the highest supported torch for python 3.7 is 1.13.1, so it looks like this is a dead end. How do I avoid this problem, since someone has successfully reviewed it? My evaluation command is as follows accelerate launch main.py \ --model bigcode/starcoder \ --tasks apps-introductory \ --max_length_generation 2048 \ --temperature 0.8 \ --n_samples 1 \ --batch_size 32 \ --save_generations \ --precision bf16 \ --save_generations_path generations.json \ --metric_output_path evaluation_results.json \ --allow_code_execution