huggingface / lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.
MIT License
471 stars 55 forks source link

Add mt-bench #75

Closed NathanHB closed 3 months ago

NathanHB commented 4 months ago

What this PR does:

This implementation uses mt-bench prompts from InflectionAI. The code is inspired from the original implementation of mt-bench with notable differences.

clefourrier commented 3 months ago

@NathanHB feel free to ping me once it's merged with main so we can integrate it :)

clefourrier commented 3 months ago

Careful with the deletion of task_examples, quite sure some of these files are needed by the nanotron team.