search

huggingface / lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.
MIT License
462 stars 53 forks source link

add openai dependency #197

Closed NouamaneTazi closed 1 month ago

NouamaneTazi commented 1 month ago

Required here https://github.com/huggingface/lighteval/blob/fd86c0c6ed6836b8ecdf3b161146ac9227c795d9/src/lighteval/metrics/llm_as_judge.py#L30

clefourrier commented 1 month ago

It's already in the optional deps extended_tasks, and we have a PR working on removing the warnings and adding tests at the metric level for this.

NouamaneTazi commented 1 month ago

Perfect! will close this then 🙌

clefourrier commented 1 month ago

Sorry, we're a bit behind on lighteval, we should be able to come back to it more from next week