search

stanford-crfm / helm

Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09110). This framework is also used to evaluate text-to-image models in Holistic Evaluation of Text-to-Image Models (HEIM) (https://arxiv.org/abs/2311.04287).
https://crfm.stanford.edu/helm
Apache License 2.0
1.8k stars 239 forks source link

HELM Leaderboard Submission Guidance Inquiry #2801

Closed gothiswaysir closed 1 week ago

gothiswaysir commented 2 weeks ago

Hello HELM Team,

I have been following this project and was very impressed by the comprehensive framework it provides for benchmarking language models. Our team is eager to submit our model to the HELM leaderboard website to evaluate its performance relative to other state-of-the-art models.

Despite thoroughly reviewing the GitHub repository, including the issues and docs file, we are still unclear on the exact steps required to make a submission to the leaderboard website.

Our model is compatible with OpenAI's API format, could you kindly provide some guidance on how we can proceed with our submission? Any additional details on the process or requirements would be greatly appreciated.

Thank you for your time and assistance.

yifanmai commented 2 weeks ago

Hi @gothiswaysir, thanks for reaching out. I'll follow up via email regarding submitting to the leaderboard.

yifanmai commented 1 week ago

Closing this thread because we followed up by email.