search

huggingface / lighteval

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
MIT License
687 stars 78 forks source link

Issue when saving results with fsspec==2023.12.1 #191

Closed Hamza-Alobeidli closed 1 month ago

Hamza-Alobeidli commented 4 months ago

Dear lighteval team,

I had this version fsspec==2023.12.1, and I had issues once the evaluation is done is failing. I am using now fsspec==2024.3.1 and it works fine. Although I did pip install -e . before running the eval_accelerate. It would be grade if you add this to the setup requirements. Thanks

clefourrier commented 2 months ago

Hi @Hamza-Alobeidli , what do you use fsspec for? We should not be needing it for lighteval.

NathanHB commented 1 month ago

Hi @Hamza-Alobeidli, just did a fresh install of lighteval to reproduce and it seems to install the 2024 version by default. closing this issue as we cannot seem to replicate. Don't hesitate to re-open if needed :)