Problem with mutliple tasks from the same dataset

huggingface / lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

MIT License

471 stars 55 forks source link

Problem with mutliple tasks from the same dataset #91

Closed clefourrier closed 3 months ago

clefourrier commented 4 months ago

Need to see if it can be repro, but I can only load BBH by manually forcing all my processes to wait for one another.

clefourrier commented 3 months ago

Was linked to force_redownload, issue was fixed in datasets here