search

open-compass / opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
https://opencompass.org.cn/
Apache License 2.0
3.73k stars 397 forks source link

[Feature] Support datasets in OpenAI/evals #542

Open Ezra-Yu opened 10 months ago

Ezra-Yu commented 10 months ago

Describe the feature

OpenAI/evals contains many community-sourced tasks, with about 400-500 datasets covering various languages and fields. Compared to other open-source datasets, they have very few questions, but many tasks come from users' real needs and inspiration when using GPT4, and many datasets give people a bright feeling.

Is there a way to directly use their datasets through a certain mechanism?

Will you implement it?

tonysy commented 10 months ago

Great idea.

merlinarer commented 8 months ago

Describe the feature

OpenAI/evals contains many community-sourced tasks, with about 400-500 datasets covering various languages and fields. Compared to other open-source datasets, they have very few questions, but many tasks come from users' real needs and inspiration when using GPT4, and many datasets give people a bright feeling.

Is there a way to directly use their datasets through a certain mechanism?

Will you implement it?

  • [ ] I would like to implement this feature and create a PR!

Hi, has any progress on this thread?