search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
39.68k stars 5.21k forks source link

Listen to OpenAI API from other devices on the network #5373

Closed DiffusionAllusion closed 6 months ago

DiffusionAllusion commented 8 months ago

Describe the bug

Hello! I'm trying to access the OpenAPI from other devices on my network. I'm running it on my Windows computer with the IP 192.168.0.33 on port 7861 (which I opened up in the Windows firewall, and port 5000).

I added the following to CMD_FLAGS.txt: --listen --listen-host=192.168.0.33 --listen-port=7861 and run .\start_windows.bat

Visiting http://192.168.0.33:7861 works on my computer, but not from other computers on the network. They just time out. I am able to ping 192.168.0.33 from other computers on the network. I even tried hardcoding the IP in the following file: https://github.com/oobabooga/text-generation-webui/blob/d8c3a5bee814f09b0868474002105dcf21a3ff1a/extensions/openai/script.py#L345

Not sure if it is related to https://github.com/oobabooga/text-generation-webui/issues/5242

Is there an existing issue for this?

Reproduction

  1. Add flags --listen --listen-host=192.168.0.33 --listen-port=7861
  2. Run start_windows.bat
  3. Try to visit the host from another computer
  4. It times out

Screenshot

No response

Logs

No errors

System Info

Windows 10. Nvidia RTX 3600.
marcg438 commented 8 months ago

Hello, you do not have to modify any files. to make it work on other devices in your network. use: start_windows.bat --api --listen and the app will load with 0.0.0.0 instead of 127.0.0.1. --api will enable the openai api it is by default at port 5000. the api port will be different than the web gui port.
image

DiffusionAllusion commented 8 months ago

I think it might be a firewall issue. I turned off the firewall to test and then it works. Apparently just adding the port to be allowed doesn't work?

marcg438 commented 8 months ago

The --api enable the openai API while --listen enable the out of local listening. You need both for API use or just --listen to use the web GUI from other computers on your network. So you need at least --listen

github-actions[bot] commented 6 months ago

This issue has been closed due to inactivity for 2 months. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.