search

RUCAIBox / LLMSurvey

The official GitHub page for the survey paper "A Survey of Large Language Models".
https://arxiv.org/abs/2303.18223
9.65k stars 745 forks source link

Suggestion: Incorporation of Hugging Face's OpenLLM Leaderboard and Results in Future Revision #57

Closed hunkim closed 8 months ago

hunkim commented 11 months ago

Dear Authors,

I recently had the pleasure of reading your "A Survey of Large Language Models" paper. The content is insightful, comprehensive, and provides a remarkable reference point for those who are interested in the world of large language models.

As an extension to the wealth of knowledge you've presented, I believe there's an additional resource that could be of great value for your readers. Hugging Face's open LLM leaderboard offers an up-to-date view on the performance and capabilities of the most current language models. The leaderboard can be found here: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard.

By including these results in your paper, I believe your readers will have a more immediate understanding of the present landscape of large language models. This addition could especially benefit readers who are keen on understanding the latest advancements and top-performing models in the field.

As a suggestion, you might also consider highlighting a few of the top models from the leaderboard. This could provide a snapshot of the current state-of-the-art, helping readers to appreciate how these models are advancing the field in real-time.

In conclusion, I'd like to reiterate how impressed I was with your work and how much I enjoyed reading your paper. I firmly believe that the inclusion of the Hugging Face leaderboard and its results would provide a significant addition to your paper's already rich content.

Thank you for considering my suggestion. I am very much looking forward to the next revision of your paper!

wxl1999 commented 11 months ago

Thanks very much for your appreciation on our work and great suggestions about the evaluation part! We will revise this part in the next version and take your suggestion into consideration.

wxl1999 commented 10 months ago

Hi, we have added Hugging Face's Open LLM leaderboard into Section 7.3.1 in the latest version. We also discuss some famous top models in Section 3.1, such as the LLaMA model family. We do not introduce the latest top models since the frequency of our update cannot catch up with the speed at which new top models are emerging.

Thanks for your great suggestion! Would you like us to acknowledge you in our survey? If so, please provide us your name.

adrianog commented 10 months ago

We do not introduce the latest top models since the frequency of our update cannot catch up with the speed at which new top models are emerging.

Pity because this is very good work, and without the evolution it will soon be out of date.