Clicking stop will erase the progress

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

https://github.com/lostruins/koboldcpp

GNU Affero General Public License v3.0

5.33k stars 363 forks source link

Clicking stop will erase the progress #1074

Open jojorne opened 3 months ago

jojorne commented 3 months ago

Describe the Issue Click stop generating before it finishes. It will erase everything that has been streamed so far. This bug has been fixed, but it is back.

Additional Information: Windows 11, Opera, Edge, Firefox, koboldcpp-1.72

LostRuins commented 3 months ago

Kobold Lite frontend? I don't suppose trim sentences is enabled?

jojorne commented 3 months ago

Kobold Lite frontend? I don't suppose trim sentences is enabled?

LostRuins commented 3 months ago

I cannot repro it. Can you link me back to the previous issue where this was resolved so I can review what was done?

LostRuins commented 3 months ago

Can you see if the latest release 1.73 still has this issue?

VL4DST3R commented 2 months ago

I've also encountered this bug on and off from time to time, although in my case it seems to lose only incomplete sentences (in all 4 modes), usually up to the previous period or new line. I also have trim sentences disabled and 'Token Streaming' set to Poll.

jojorne commented 2 months ago

Every time it outputs a , with it as a stop seq, text generated disappear.

Edit: I have XTC and DRY on.

LostRuins commented 2 months ago

Every time it outputs a , with it as a stop seq, text generated disappear.

Do you mean all the text after and including the , disappears? Or everything in the output disappears?

jojorne commented 2 weeks ago

My quick test prompt:


{{[INPUT]}}
I need you to continue the following list:

**Fruits**

banana, apple,
{{[OUTPUT]}}
Here is the continued list:

**Fruits**

I think I found the reason: Set stop sequence to ',' or even 'apple' and then usage mode to "Instruct Mode". It will trigger the end_trim_to_sentence cleaning the whole paragraph. I will fix on my end by querying the stop sequence. On "normal" operation, it fails to find the "/n" resulting in last "-1". But yeah, certain modes will trigger this behavior by default. So trying another mode also solves the problem.

VL4DST3R commented 2 weeks ago

I don't think you should close this, maybe an official fix should/could be implemented since multiple people encountered this who won't know how or why it happens.