Add BBH subset back! - Githubissues

huggingface / lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

MIT License

467 stars 54 forks source link

Add BBH subset back! #125

Closed clefourrier closed 3 months ago

clefourrier commented 3 months ago

The BBH subsets (lighteval and harness versions) were removed in this merge commit https://github.com/huggingface/lighteval/pull/7/commits/c136ad59fc74bb3eee6546dcf0802eb8c2f3bcbe. They need to be added back to the table, and the test set need to use BBH again (need to uncomment the call to BBH)