💡 [REQUEST] - <title> Could you add the evaluation of ConBench. - Githubissues

QwenLM / Qwen-VL

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

Other

4.24k stars 326 forks source link

💡 [REQUEST] - <title> Could you add the evaluation of ConBench. #405

Open Gumpest opened 1 month ago

Gumpest commented 1 month ago

起始日期 | Start Date

No response

实现PR | Implementation PR

No response

相关Issues | Reference Issues

No response

摘要 | Summary

The ConBench is from https://github.com/foundation-multimodal-models/ConBench, and we find Qwen-VL-Max leads the board. Do you have an interest in incorporating Conbench?

基本示例 | Basic Example

Rank	Teacher	ConScore[D]
1	Qwen-VL-Max	37.00
2	GPT-4-Omni	35.70
3	InternVL-v1.2P-40B	34.70
4	Gemini-Ultra-Vision	33.10
5	InternVL-v1.5-26B	31.40

缺陷 | Drawbacks

-

未解决问题 | Unresolved questions

No response

Gumpest commented 1 month ago

@hzhwcmhf @tinytangent Hi guys. Could you add the evaluation of ConBench.