search

EleutherAI / lm-evaluation-harness

A framework for few-shot evaluation of language models.
https://www.eleuther.ai
MIT License
6.41k stars 1.69k forks source link

RecursionError: maximum recursion depth exceeded while calling a Python object #2121

Open yaolu-zjut opened 1 month ago

yaolu-zjut commented 1 month ago

When I run the code "CUDA_VISIBLE_DEVICES=3 TRANSFORMERS_OFFLINE=1 lm_eval --model hf --model_args pretrained=/public/MountData/yaolu/LLM_pretrained/LLAMA2_7B/,trust_remote_code=True --tasks mmlu,cmmlu --device cuda:0 --batch_size auto --num_fewshot 0" for llama2-7b, it reports "RecursionError: maximum recursion depth exceeded while calling a Python object". However, it works well on llama3-8b and baichuan2-7b? So how can I fix this problem.

yaolu-zjut commented 1 month ago

By the way, the version of lm-eval is 0.4.3

haileyschoelkopf commented 1 month ago

What Llama 2 download is this?

Could you post the entire traceback?