search

felipemaiapolo / tinyBenchmarks

Evaluating LLMs with fewer examples
MIT License
107 stars 11 forks source link

The correspondence between the predictions data and model #8

Closed marvelcell closed 6 days ago

marvelcell commented 6 days ago

Thank you for your outstanding work, it really helps me a lot!

I notice that you've released the predictions data of the models you mentioned in paper, but I can't find the corresponding relationship between the data and the model, and I don't know which prediction data comes from which model. Could you please explain that?

Thank you very much, and thank you for your excellent work again!

LucWeber commented 6 days ago

Hey Zhang Yueqi,

happy to hear that it is helpful! In the pickle files, the ordering of the list of models (under the 'model' key) corresponds to the ordering of model dimension in the correctness data. Let me know if that resolves your question.

Best, Lucas

marvelcell commented 6 days ago

Thank you for your fast reply, Your answer solved my problem！