search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
8.89k stars 489 forks source link

Petals server Network ports remain open after python process shutdown #509

Open redcap3000 opened 9 months ago

redcap3000 commented 9 months ago

After shutting down a petals server in suggested fashion (ctl-c style interrupt) the port used for the server remains open when the port is properly forwarded to the public internet. Failed requests to these open ports also occur a few min. after shutdown, causing noticeable slowdown for other non petals uses of the router until the open ports are closed.

Using Ubuntu Server 23.04, 32gigs of ram, merlin-wrt software on asus RT AX86U router connected via 2.5 ethernet to 2 gig fiber. Petals installed without use of docker.

borzunov commented 9 months ago

Hi @redcap3000,

Does it only happen when you use KillSignal=SIGINT in systemd? If yes, please don't use this option for now - probably it doesn't shut down all child processes correctly.