search

QwenLM / Qwen2

Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud.
7.3k stars 442 forks source link

qwen1.5-14b-chat performance is worse than 1.0 #210

Open kxleee opened 5 months ago

kxleee commented 5 months ago

It is not as useful as qwen1 in real use cases and is not a generic benchmark

Math skills are no good

eg. 去年阿里的营收534785万元，腾讯的是54787万元，这两个公司哪个的营收比较多，高多少

use：vllm 0.3.3

lissettecarlr commented 5 months ago

qwen-7B：根据你提供的数据，阿里巴巴的营收为534785万元，腾讯的营收为54787万元。因此，阿里巴巴的营收比腾讯的营收多，多出13092万元。 qwen1.5-7B：根据您提供的数据，去年阿里巴巴的营收为534,785万元，而腾讯的营收为54,787万元。比较这两个数字，阿里巴巴的营收明显更高。要计算高多少，用阿里巴巴的营收减去腾讯的营收：534,785万元 - 54,787万元 = 479,998万元阿里巴巴的营收比腾讯高479,998万元。 qpt3.5：去年阿里的营收为534785万元，腾讯的营收为54787万元。因此，去年阿里的营收比腾讯的营收多了479998万元。

后面两个都计算对了

kxleee commented 5 months ago

I gave you just one example, and you answered my question very cleverly

thorory commented 5 months ago

qwen1.5-7B：根据你提供的数据，阿里巴巴的营收明显要多于腾讯。阿里巴巴的营收为534,785万元，而腾讯的营收为54,787万元。要计算高多少，我们需要做简单的减法：\n\n534,785 - 54,787 = 479,998（万元）\n\n所以，阿里巴巴比腾讯多479,998万元。 qwen1.5-14B：去年阿里巴巴（Alibaba）的营收为534,785亿元人民币，而腾讯（Tencent）的营收为54,787亿元人民币。通过比较，阿里巴巴的营收要高于腾讯。\n\n两者之间的差距是：\n\n534,785亿元 - 54,787亿元 = 480,000亿元\n\n所以，阿里巴巴的营收比腾讯高480,000亿元人民币。

为什么测了几个例子，感觉14B的模型还明显不如7B。。

github-actions[bot] commented 2 months ago

This issue has been automatically marked as inactive due to lack of recent activity. Should you believe it remains unresolved and warrants attention, kindly leave a comment on this thread.