ModuleNotFoundError: No module named 'lighteval'

[xinghuang2050]

Hi. I have successfully installed lighteval package.

When I run the example task: accelerate launch --multi_gpu --num_processes=8 run_evals_accelerate.py --tasks="leaderboard|hellaswag|5|1" --output_dir ./evals/ --model_args "pretrained=./ablation-model-fineweb-edu/"

it returned the ModuleNotFoundError:

from lighteval.main_accelerate import CACHE_DIR, main ModuleNotFoundError: No module named 'lighteval'

Python: Python 3.11.5 Packages list: Package Version

---

absl-py 2.1.0 aenum 3.1.15 aiohttp 3.9.5 aiosignal 1.3.1 annotated-types 0.7.0 anyio 4.4.0 archspec 0.2.1 attrs 23.2.0 blis 0.7.11 boltons 23.0.0 Brotli 1.0.9 catalogue 2.0.10 certifi 2023.11.17 cffi 1.16.0 chardet 5.2.0 charset-normalizer 2.0.4 click 8.1.7 cloudpathlib 0.16.0 colorama 0.4.6 conda-content-trust 0.2.0 conda-package-handling 2.2.0 conda_package_streaming 0.9.0 confection 0.1.5 cryptography 41.0.7 cymem 2.0.8 DataProperty 1.0.1 datasets 2.19.2 dill 0.3.8 distlib 0.3.8 distro 1.8.0 filelock 3.13.4 frozenlist 1.4.1 fsspec 2024.3.1 gitdb 4.0.11 GitPython 3.1.43 h11 0.14.0 httpcore 1.0.5 httpx 0.27.0 huggingface-hub 0.23.3 idna 3.4 Jinja2 3.1.4 joblib 1.4.2 jsonpatch 1.32 jsonpointer 2.1 langcodes 3.4.0 language_data 1.2.0 libmambapy 1.5.3 lighteval 0.4.0.dev0 lxml 5.2.2 marisa-trie 1.2.0 MarkupSafe 2.1.5 mbstrdecoder 1.1.3 menuinst 2.0.1 mpmath 1.3.0 multidict 6.0.5 multiprocess 0.70.16 murmurhash 1.0.10 networkx 3.3 nltk 3.8.1 numpy 1.26.4 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 8.9.2.26 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu12 2.20.5 nvidia-nvjitlink-cu12 12.5.40 nvidia-nvtx-cu12 12.1.105 openai 1.32.0 packaging 23.1 pandas 2.2.2 pathvalidate 3.2.0 pip 23.3.1 platformdirs 3.10.0 pluggy 1.0.0 portalocker 2.8.2 preshed 3.0.9 protobuf 3.20.3 pyarrow 16.1.0 pyarrow-hotfix 0.6 pycosat 0.6.6 pycountry 24.6.1 pycparser 2.21 pydantic 2.7.3 pydantic_core 2.18.4 pyOpenSSL 23.2.0 PySocks 1.7.1 pytablewriter 1.2.0 python-dateutil 2.9.0.post0 pytz 2024.1 PyYAML 6.0.1 regex 2024.5.15 requests 2.32.3 rouge-score 0.1.2 ruamel.yaml 0.17.21 sacrebleu 2.4.2 safetensors 0.4.3 scikit-learn 1.5.0 scipy 1.13.1 sentencepiece 0.2.0 setuptools 68.2.2 six 1.16.0 smart-open 6.4.0 smmap 5.0.1 sniffio 1.3.1 spacy 3.7.2 spacy-legacy 3.0.12 spacy-loggers 1.0.5 srsly 2.4.8 sympy 1.12.1 tabledata 1.3.3 tabulate 0.9.0 tcolorpy 0.1.6 termcolor 2.3.0 thinc 8.2.4 threadpoolctl 3.5.0 tokenizers 0.19.1 torch 2.3.1 tqdm 4.65.0 transformers 4.41.2 triton 2.3.1 truststore 0.8.0 typepy 1.3.2 typer 0.9.4 typing_extensions 4.12.1 tzdata 2024.1 urllib3 1.26.18 virtualenv 20.25.3 wasabi 1.1.3 weasel 0.3.4 wheel 0.41.2 xxhash 3.4.1 yarl 1.9.4 zstandard 0.19.0

[xinghuang2050]

I can import lighteval package in the python command, it's really funny.

[clefourrier]

Hi! This feels like a mismatch between two environments, did you make sure to 1) install everything in a specific environment (using conda for example) and 2) that said environement was well activated in both situations?

[barneylogo]

hell @clefourrier nice to meet you may I have question? for running nanotron script, I got this result I'd like to evaluate this using lighteval but it is not working on my side. I appreciate if you help me

[clefourrier]

Hi @barney49 , please open a dedicated issue for your problem, and tell us there which version of lighteval you're using, which command you're launching, with which model, etc.