search

ViperX7 / Alpaca-Turbo

Web UI to run alpaca model locally
GNU Affero General Public License v3.0
876 stars 92 forks source link

Long generations are cut off in the webui #36

Open Lesani opened 1 year ago

Lesani commented 1 year ago

with standard settings and both models I have - alpaca 7b and 13b-gpt-x longer generations are cut off in the webui. image

after a while text stops appearing, the debug console shows only status messages, no more "polling" messages, but CPU usage stays up and the UI also shows the "stop generating" button

upon pressing that button a minute later, the console shows the much longer message (it was still generating), the message is not shown in the web interface

image

ViperX7 commented 1 year ago

can you provide entire transcript

i suspect that this issue is related to windows somehow

actually i have pushed some changes to my fork of llama.cpp i think they might fix the issue

fblgit commented 1 year ago

There is something weird... when post like a 'medium-size' text.

Immediately see this:

inp( #2) :  inp( #2) :  inp( #2) :  inp( #2) :  inp( #2) :  inp( #2) :  inp( #2)

then the answer...

answer for the question

followed by a loop

### Human: continue
### Assistant: Additionally, ...

I find a bit concerning this behaviour on many new models, where the stream gets unlimited by just these human/assistant loops..

ViperX7 commented 1 year ago

This should be fixed in the latest release please check and confirm