search

g-simmons / persona-research-internship

0 stars 0 forks source link

Ques: Are ontology scores correlated with model performance on OpenLLM Leaderboard? #157

Open g-simmons opened 4 weeks ago

g-simmons commented 4 weeks ago

Background

We are curious to know whether ontology score correlates to performance on downstream tasks.

We could evaluate performance on downstream tasks ourselves, but as a first approximation, we could use results that have already been obtained for the models.

Open LLM Leaderboard reports model performance on a variety of tasks, and has results for Pythia models. Let's see whether there is any correlation between ontology score and Open LLM Leaderboard performance.

Tasks

g-simmons commented 4 weeks ago

@ranr9131 Can you elaborate on where someone can find the ontology scores, and how to get scores from OpenLLM leaderboard?