Open guanqun-yang opened 9 months ago
@arjunguha It seems that #88 is not merged into the main branch but is somehow closed. Running the code taken from the HuggingFace data card returns errors:
main
python3 main.py --model bigcode/gpt_bigcode-santacoder --tasks studenteval --max_length_generation 512 --n_samples 20 --batch_size 20 --precision bf16 --allow_code_execution
Is there any way I could run the evaluation on the StudentEval dataset any time soon?
whoops. mistake. let me try to merge it in this time. i can probably get it running in a day or so
@arjunguha It seems that #88 is not merged into the
main
branch but is somehow closed. Running the code taken from the HuggingFace data card returns errors:Is there any way I could run the evaluation on the StudentEval dataset any time soon?