No errors, extras API unreachable

[Yo1up]

I recently installed the extras API according to the install instructions on the readme. I am running the application in an Ubuntu VM with 12 cores and 24GB RAM.

The server file runs without errors however it is unreachable or unresponsive from localhost:5100 as sillytavern itself cannot connect to the extras API.

I attempted reinstalling it, I attempted to allow external port listening and ingress from an external port, and I attempted putting in an API key despite it saying it doesn't need one. none of these have fixed the issue, and I have const enableExtensions = true

I would like to emphasize that there are no errors at all. the entire console output is the following:

python server.py --enable-modules=websearch
Using torch device: cpu
No API key given because you are running locally.
 * Serving Flask app 'server'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://localhost:5100
Press CTRL+C to quit

when I ping localhost it uhh pings back. I honestly don't know what kind of information would be helpful, just let me know what you need.

[Cohee1207]

Consider adding --listen flag

[Yo1up]

I did that. That was what I meant when I said "I attempted to allow external port listening and ingress from an external port" and unfortunately that did not work.

edit: here is the console

python server.py --enable-modules=websearch --listen
Using torch device: cpu
No API key given because you are running locally.
 * Serving Flask app 'server'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5100
 * Running on http://192.168.1.148:5100
Press CTRL+C to quit

and here is a screenshot of sillytavern

[Cohee1207]

1. Open a port in the firewall.
2. Set the network to private.

[Yo1up]

I used sudo ufw allow 5100 and it still doesn't work, I have a workaround that seems to function but I would like to continue this to figure out what is happening. my workaround is to use the tailscale IP of the server which works for some reason.

and I suppose I don't know how to set the network to private, I assumed since it was in a VM it would already be configured as such.

[mrguymiah]

1. Open a port in the firewall.
2. Set the network to private.

I've got a similar issue where the logs show it's open to communication, but it just isn't. For context, I'm running it as a docker container. So Windows Firewall isn't in the way. It shows as listening and lists its docker virtual network IP. When attempting to connect with SillyTavern, it just doesn't. I also enabled the "--secure" flag and put in the API key.

EDIT: Interestingly, when I utilize the "--share" flag and utilize a Cloudflared connection, THAT works.

[mrguymiah]

Final thing before I go to bed. Interestingly, testing from a simple REST API tool on my workstation, I'm able to communicate with the container from the host side. Checking the logs, I see the successful connections across the Cloudflare setup, and I see my failed connections (I don't know the proper header to supply the key with, or if it needs encrypted). What I don't see is failed attempts to connect from the SillyTavern Instance from the docker network of 172.19.0.0/24.

I'm shutting it down (don't trust Cloudflare's free option as secure) and going to bed.

[Cohee1207]

Pass the API key as Authorization header Bearer ${apiKey}

[mrguymiah]

Thanks for the Syntax, Cohee.

Using that, I was able to confirm communication over a local port from my API tester. Still not establishing connections from SillyTavern to ST-E. We've got connections to SillyTavern-Extras, so I'm not sure it's an ST-E issue. Perhaps I should open an issue over on SillyTavern's side? Or would you prefer I keep it here for cleanliness, @Cohee1207

^The last error is caused by me attempting an API GET for SD models despite that module not being loaded. As it does not show successful connections, I thought that a simple way to show that I was connected.

[Cohee1207]

But that key into SillyTavern's UI in the extensions panel, together with the URL.

[mrguymiah]

Issue persists when using the container name or the docker network IP.

[Cohee1207]

Here's the problem: Extras API must be accessible from the client's browser. The requests are done directly from there, not proxied via the server.

[mrguymiah]

That there is the trick. SillyTavern proxies the LLM Backend connection, but not the extras. So, for my model, I've got the Ooba Booga backend utilizing the container name. I expected the same from the extras API. Now that I'm using the host's IP (which proxies the open port to the container), I'm reaching it from Silly Tavern. It is an issue that it means I'll need to publicly expose the SillyTavern-Extras instance to be able to be able to utilize it from outside my home. Hm... Not sure the best way to secure that. Any suggestions beyond usage of a LOOOONG API key and port obfuscation?