qwen 14b 不微调的情况下，问相同的问题，模型输出也不太一致，是为什么？温度已经设置成0了

[Wangqi12138]

          > > 我也遇到了，好像确实都不行，同求解

好像不微调也会，没搞清楚怎么引起的

想问下这个问题解决了吗？

Originally posted by @Elissa0723 in https://github.com/QwenLM/Qwen/issues/1025#issuecomment-1960662030

[jklj077]

Hi!

It appears that the original author was using a quantized model. Unfortunately, most efficient kernels for GPTQ or AWQ use non-deterministic algorithms, such that the results may be slight different even when do_sample is set to False. In addition, transformers does not accept temperature=0 in recent versions and it will ask you to use do_sample=False.

However, it is unclear that whether your issue is the same with the one you have referenced. Please describe with more details, such as which model were you using and which framework were you using.

[jklj077]

As Qwen1.0 is no longer actively maintained, we kindly ask to you migrate to Qwen1.5 and direct your related question there. Thanks for you cooperation.