search

vicuna-tools / vicuna-installation-guide

The "vicuna-installation-guide" provides step-by-step instructions for installing and configuring Vicuna 13 and 7B
285 stars 34 forks source link

Different and sometimes wrong answers with ggml-vic13b-q5_1.bin + ggml-vic13b-uncensored-q5_1.bin #8

Closed breisig closed 1 year ago

breisig commented 1 year ago

After running the latest instructions today to install it, I noticed when trying to run and use ggml-vic13b-q5_1.bin [ ./main -m models/ggml-vic13b-q5_1.bin -f 'prompts/chat-with-vicuna-v1.txt' -r 'User:' --temp 0.36 ] or ggml-vic13b-uncensored-q5_1.bin.

However, every time I stop the app and run it again and ask it the same question, I can get different + wrong answers. For example

User:What is the closest planet to earth?
Vicuna: The closest planet to Earth is Venus, which is about 0.38 AU (5.1 million km or 3.2 million miles) away from Earth on average.

is fine but if I close + start running the app again about 5-8 times, I'll get a different/wrong answer.

User:What is the closest planet to earth?
Vicuna: The closest planet to Earth is the Moon.

Is it normal for ... 1) Get different answers asking the same question after a restart? 2) It gives wrong answers?

fredi-python commented 1 year ago

Hi, Yes its normal if u get different answers. They are also might be wrong but that's not a issue of the installation guide, as vicuna isn't my language model.

gerardofisch commented 1 year ago

This is due to the temperature parameter set to 0.36. The temperature parameter is used to control the randomness of the model's output. The value is between 0.0 and 1.0. A higher temperature value, such as 1.0, increases the randomness, while a lower value, such as 0.0, reduces it.

When the temperature is set to 0.0, the model tends to produce deterministic responses. In other words, it will consistently generate (almost) the same output for a given input.