Fix parallel data processing bug

huggingface / lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

MIT License

471 stars 55 forks source link

Fix parallel data processing bug #92

Closed clefourrier closed 4 months ago

thomwolf commented 4 months ago

fixing it on my side, thanks!

clefourrier commented 4 months ago

@thomwolf feel free to approve so I can merge it :)