search

PygmalionAI / gradio-ui

Prototype UI for chatting with the Pygmalion models.
GNU Affero General Public License v3.0
225 stars 66 forks source link

Infinite loading #10

Closed 0x000011b closed 1 year ago

0x000011b commented 1 year ago

Some users are reporting the the Gradio UI glitches out and just loads forever sometimes. I've been told this is a common thing with Gradio apparently, but I also suspect it might be because of this loop:

https://github.com/PygmalionAI/gradio-ui/blob/ee3855035373a227aa8b8edb3138fb3076439675/src/koboldai_client.py#L76

The only stopping conditions are having generated enough tokens, or having reached a \nYou:. I'm not certain but I believe that it might be possible for the model to get stuck generating an EOS token for example, so it'd never meet any of those.

Investigate and consider adding a "give up" mechanism after enough attempts at completing the generation.

0x000011b commented 1 year ago

Seems like I was wrong. Caught this one for myself finally and it does seem to be a Gradio bug. Specifically, it seems that the tunnel dies and the server becomes unreachable from the user's browser, and the front-end doesn't handle that cleanly and just loads forever instead.

Unfortunate and I don't have the time to dig deeper into this, so I'll be closing as a wontfix since my focus is on the official UI now.