search

mlfoundations / datacomp

DataComp: In search of the next generation of multimodal datasets
http://datacomp.ai/
Other
642 stars 54 forks source link

Frequency of Leaderboard Updates #76

Closed brunnedu closed 8 months ago

brunnedu commented 8 months ago

Hey there,

We submitted to the leaderboard 2 days ago, and although the evaluation code ran successfully with the --submit flag, we are unsure if the leaderboard simply hasn't been updated yet or if there were any issues with our submission.

Also, we are wondering if the evaluation is re-run on your side or if the evaluation results from our local runs are used.

gabrielilharco commented 8 months ago

Hi @brunnedu, we got your submission and the leaderboard should be updated now. The evals are not re-run for now, but we may do so in the future.