search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.58k stars 5.31k forks source link

it cannot run 2 text generation in the same time #1807

Closed ChrisXULC closed 1 year ago

ChrisXULC commented 1 year ago

when i try to access the web UI in two browsers and input some text, I found that their output is sequential and I have to wait for one to finish generating before the other can start.is there a way to make them work at the same time? llama.cpp

talvasconcelos commented 1 year ago

I also would like to "serve" this as a service for other people! I think --public-api fixes some of the issues but it would be nice to be able to clarify with someone how far it can be used!

I was able to use the webui and tavernAI (with the API), at the same time but not sure if this can handle any more connections!

github-actions[bot] commented 1 year ago

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.