LLM models have bad output

[kalous12]

This is my models Qwen1.5-0.5B the models website is https://huggingface.co/Qwen/Qwen1.5-0.5B

I have to use this to convert rkllm,the convert output is :

my program printf is

when i run the model it give me so werid answer,such as :

[fydeos-alex]

Reasonable output. It's a 0.5B model!!! 😆

[kalous12]

Reasonable output. It's a 0.5B model!!! 😆

no i have tried Qwen 1.8B MiniCPM 2B and TinyLLAMA 1.1B They all have bad output. such a sad thing

[lzz773751548]

Are you using the correct prompt?

[fydeos-alex]

Well, there could be something wrong with your model convert because Qwen 1.8B's response is much better and fluent. However, you should not count on the small language model to answer your question as GPT does.

[kalous12]

Are you using the correct prompt?

I use the default prompt,i have no chage in main.cpp , but there have some werid thing when i first use main.cpp. I complie main.cpp and I run it,It can not run and it print the env have no opencl library(the defaullt set "use_gpu=false") I download the libmali library and soft link to the opencl library.main.cpp can run and have some werid output. I have tried rknn-llm(v1.0.0) and it can run so well.So that make me so werid!!

[kalous12]

Well, there could be something wrong with your model convert because Qwen 1.8B's response is much better and fluent. However, you should not count on the small language model to answer your question as GPT does.

can you try to convert qwen_0.5 and run it or share the Qwen 1.8B which you convert with me

[fydeos-alex]

I am sorry, but I am still using the version v1.0.0. I will try the new version later and share the latest information with you.

[kalous12]

thank you, waiting for your good news

[SunAriesCN]

The latest version Qwen-1.8B can output right answer with the example codes.