Repetitions and hallucinations in Windows and Web versions

kwaroran / RisuAI

Make your own story. User-friendly software for LLM roleplaying

https://risuai.net

GNU General Public License v3.0

793 stars 140 forks source link

Repetitions and hallucinations in Windows and Web versions #634

Closed KingXMAX closed 1 month ago

KingXMAX commented 1 month ago

I use the API versions of Mistral Large and AI21 Jamba Large models on RisuAI Windows and Web versions, and recently, i notice that most responses are repetitive and sometimes there are very long responses that contain hallucinations. I set the Temperature to 0.80 and repetition penalty to 1.15 and Top P to 1 but to no avail. I followed the common instructions found on internet to reduce the issue but the problem persists. So, How can i solve this problem? Is this a bug in RisuAI itself (Because things work well in another popular app)?

Asherathe commented 1 month ago

I haven't used those specific models, but in general I haven't experienced these issues in RisuAI. I would say that responses do tend to be longer than in, say, Kobold Lite or SillyTavern. But, IMO, the quality of the writing is better, without messing with any settings.

KingXMAX commented 1 month ago

I think i disagree with you. I have set the same parameters in SillyTavern, yet it gives better responses that i can control regarding length and style, while RisuAI keeps repeating the responses and giving low quality answers no matter how i adjust the parameters and prompts. it seems like ST is much better than RisuAI, and if the dev here wants get ahead of other apps, then he/she must fix this issue for better usability and smoother experience for RisuAI users. NOTE: I have tried numerous times before i post this>>>RisuAI rating: 4/10>>>ST rating: 8/10

Asherathe commented 1 month ago

You raised the possibility that there is a bug in the program. All I can say is that I haven't seen any evidence of that. It wasn't my intention to say it is better or worse overall than any specific alternative. That, of course, is subjective.

KingXMAX commented 1 month ago

Well, you've got a point here and i think it may not be necessarily a bug, but do you think RisuAI would perform better and give better and more coherent responses with the android app? I would like to know before i install it on my phone so if you tried it on android, feel free to share your experience.

kwaroran commented 1 month ago

The probability of this being a bug is very low, and in fact, it is safe to say that there is no probability.

This is based on the operating principle of the "frontends".

The frontend does not operate the AI model directly. Instead, it requests the server that operates the AI model to operate.
The frontend sends "parameters" and "prompts" to the server that operates the AI. Except for very few cases, the frontend cannot control other data, and the only things it can control are "Stop Strings" and "Logprobs" that do not affect the quality of the answer.
Then, the "parameters" and "prompts" can go wrong, and both ST and Risu leave this up to the user's settings.
However, Risu and ST have different default settings. If you feel it has low quality, try using a different prompt or adjusting the parameters.

KingXMAX commented 1 month ago

Okay, i have crafted a new prompt with the help of a custom GPT, and i put that prompt in RisuAI and i set the parameters all over again, and now the responses are better and repetitions are much less, finally. Thank you for your appreciated help, the issue can now be closed.