[Tracker] OpenRouter LLM rankings tracking

zhyncs commented 3 weeks ago

Checklist

[X] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
[X] 2. Please use English, otherwise it will be closed.

Motivation

This issue is not a bug report or feature request, but just a record of tracking the current popular LLM.

From https://openrouter.ai/rankings Top this month, it can be known that, currently in addition to using the closed-source model APIs of OpenAI, Anthropic and Google, users also use open source LLM. The difference between OpenRouter and other rankings is that it better reflects the real usage scenarios and actual conditions of users.

The list shows that the currently mainly used series are Llama 3, Llama 3.1, Gemma 2, Mistral, Mixtral, and DeepSeek Coder V2. Besides these mentioned models, there are some based on these models through SFT, for example, MythoMax 13B is based on Llama 2, and WizardLM-2 8x22B is based on Mixtral 8x22B. These popular models are all supported in SGLang at present(August 19, 2024).

Related resources

No response

zhyncs commented 3 weeks ago

If there are open source LLMs being used in real production environments, please note that here we refer to the production environment (usually an enterprise-level data center), and it is not yet supported in SGLang. You are welcome to raise this issue here, and we will prioritize its support. Of course, it would be best if you could explain why your company (or team) needs support for this open source LLM and what scenarios you use it for.

zhyncs commented 1 week ago

ref https://github.com/sgl-project/sglang/issues/1283

note:

US: Llama 3.1, Gemma 2, Phi 3 Europe: Mixtral China: DeepSeek 2, Qwen 2

sgl-project / sglang