OpenGenerativeAI / llm-colosseum

Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
https://huggingface.co/spaces/junior-labs/llm-colosseum
MIT License
1.34k stars 160 forks source link

Is there a way to set it to do best 3 of 5? #40

Open lafintiger opened 8 months ago

lafintiger commented 8 months ago

Is there a way to get it to set the matches to a best of... 3 of 5 4 of 7, etc

Thanks,

oulianov commented 8 months ago

You should ask about it on the Diambra discord ! https://discord.gg/YduaNe5cN8

lafintiger commented 8 months ago

I checked with them. The answer is basically no. So it would have to be part of the script here and just tell it to loop, checking the results.csv file and getting out when the number is reached. I will see what I can cook up.

oulianov commented 8 months ago

That works as well 👍