Ignores new query and responds with crossed out details (from previous question).

liltom-eth / llama2-webui

Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.

MIT License

1.97k stars 202 forks source link

Ignores new query and responds with crossed out details (from previous question). #54

Closed THREELabs closed 1 year ago

THREELabs commented 1 year ago

Hi, testing this great tool. Noticed bot ignored last questions and responded with previous question details, along with it crossed out. Have any idea why? Could it just be the model? Maybe not supported, tried it anyway. Tried -> Wizard-Vicuna-13B-Uncensored.ggmlv3.q2_K

liltom-eth commented 1 year ago

@THREELabs interesting, from the backend, the model will load all chat history into this kind of text:

[INST] <<SYS>>
You are a helpful, respectful and honest assistant. 
<</SYS>>

{prompt} [/INST] {model_reply} [INST] {prompt} [/INST]

Thus it will remember all history until exceed the limitation. If you change the topic, better to start a new conversation. It also depends on models I believe.

Will also have a look at these symbols issues: [INST]..., it is weird that showing a lot [INST]... in your chat history.

THREELabs commented 1 year ago

Yeah I think does have to do with the model as well. I've thrown a handful at it and some do that more than others. Not really a problem I guess if I can change topic properly and or clear chat.