Open merplumander opened 2 weeks ago
Logprobs: yes
Logprobs: no
Problem: Knowledge cut-off information not available
Logprobs: No
grok-beta
No further information available without X-premium.
No logprobs from api available.
Very good documentation for the open source downloadable models, but almost none for commercial models available via the API. It would be great to run the open source versions but they are too large without additional resources.
Based on the information above we can do an evaluation with data starting from August (the 1st):
Language Model Overview
OpenAI
Logprobs: yes
Anthropic
Logprobs: no
Gemini
Problem: Knowledge cut-off information not available
Logprobs: No
Llama
Grok
grok-beta
No further information available without X-premium.
Mistral
No logprobs from api available.
Qwen
Very good documentation for the open source downloadable models, but almost none for commercial models available via the API. It would be great to run the open source versions but they are too large without additional resources.