search

toverainc / willow-inference-server

Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS
Apache License 2.0
384 stars 35 forks source link

Allow to specify the location of TLS certificate and host name to use. #86

Open obones opened 1 year ago

obones commented 1 year ago

Right now, the run.sh script assumes key.pem and cert.pem live next to it and that they apply to the hostname of the machine where the container will start. However, this may not always be the case, especially when the server is located in a DMZ with NAT allowing it to be reached via different names. In my case, the machine is named server but is only reachable via willow.domain.com And for this FQDN, I already have a Let's Encrypt certificate that matches and is used by various other services.

I know I could use a symbolic link that points towards the appropriate files, but that means a reliance on filesystem functionality when I already have an environment file that sets the appropriate values for other services. It would thus be really nice if one would be able to specify the location of the two certificate files from the .env file that is sourced into the script. And for the certificate to be accepted, I believe the process should also be told to expect connections from willow.domain.com and the machine hostname (server)

Note that removing TLS requirements altogether as in #82 would also be good in my case as both WIS and the Willow device live in the same private and secluded network and would happily talk to each other without any access to the outside world.

kristiankielhofner commented 1 year ago

For configurations such as this we currently recommend using TLS with the frontend/proxy you're currently using to manage certificates (such as Traefik, another nginx instance, etc) and they connect that frontend to WIS via HTTP.

This isn't ideal from a networking standpoint but many users have already implement frontends such as this for their HA instance (as one example).

That said this is pretty basic functionality that should be supported. However, to my knowledge nginx doesn't natively allow for the configuration file to access environment variables. We would have to add another step using envsubst or something like it to generate the config file on the fly from a template with environment variables.

obones commented 1 year ago

Yes, I'd love to do just that actually (via Apache as it's the one I'm already using). But from what I read in #82, it's not already available, only in the wisng branch which I have no information on its stability. Am I right here?

kristiankielhofner commented 1 year ago

Yes, it's only currently available in wisng but we've done a lot of testing internally and with a few users. It's very stable but it hasn't been merged to main because we haven't quite tested as much as I would like and it's still developing fairly rapidly.

obones commented 1 year ago

Well, if you don't mind, I'll stay on the "master" branch until I have figured out my "stuck in P0" issue.

kristiankielhofner commented 1 year ago

No worries. The actual WIS config hasn't changed that much in wisng, the big change is the nginx frontend for socket handling.

obones commented 11 months ago

I have just pulled the latest main and I believe the wisng branch has been merged into it but I still have the issue with using my Let's Encrypt certificate via symbolic links. I was able to use utils.sh run after having done those changes:

  1. Comment out the chmod line inside utils.sh that tries to set 0666 on the certificate files
  2. Make sure that group can read/execute the /etc/letsencrypt/archive and /etc/letsencrypt/live directories.
  3. Add a new volume in docker-compose.yml that maps /etc/letsencrypt/ inside the container: /etc/letsencrypt:/etc/letsencrypt

I'm not sure this is enough, or meant to be used like that so please let me know