Question about the paper

Psycoy / MixEval

The official evaluation suite and dynamic data release for MixEval.

https://mixeval.github.io/

196 stars 28 forks source link

Closed felipemaiapolo closed 2 months ago

felipemaiapolo commented 2 months ago

Hi all,

Thanks for the nice work!

I had some questions regarding the paper:

Could you give me more details about the length constraint you used when selecting questions based on similarities?
Is the evaluation data available anywhere (the correctness scores for mixEval for the specific models used in your paper)?

Thank you!

Psycoy commented 2 months ago

Hi Felipe,

Thanks for your questions!

Could you give me more details about the length constraint you used when selecting questions based on similarities?

Is the evaluation data available anywhere (the correctness scores for mixEval for the specific models used in your paper)?

Sorry, we didn't plan to keep that. But if you have GPUs, it's quick and simple to get the desired outputs by running the models you want, the configs are all set!