Running in SystemD for boot startup

toverainc / willow-inference-server

Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS

Apache License 2.0

352 stars 31 forks source link

Running in SystemD for boot startup #102

Closed maxi1134 closed 12 months ago

maxi1134 commented 12 months ago

Hey there!

I am trying to get the inference server to start on boot on my ML machine.

I tried creating the following service


[Unit]
Description=Willow service

[Service]
ExecStart=/home/maxi1134/willow-inference-server/run.sh

[Install]
WantedBy=multi-user.target

but get the error Jul 17 20:28:38 machinelearning run.sh[183367]: the input device is not a TTY

Any help would be appreciated!

nikito commented 12 months ago

If your machine is running a normal docker daemon/service, then if you do run.sh -d it should run automatically even with reboots, no need for a separate service. Run.sh is just calling docker run. 🙂

EDIT: I just realized I am thinking of the wisng branch of WIS. The main branch doesn't allow arguments to be passed. May want to give the wisng branch a shot!

maxi1134 commented 12 months ago

The WISNG branch seems to work!

Thanks!

nikito commented 12 months ago

Just a fyi looks like wisng was merged into main. 🙂