encode / uvicorn

An ASGI web server, for Python. 🦄
https://www.uvicorn.org/
BSD 3-Clause "New" or "Revised" License
8.11k stars 701 forks source link

feat: Improve process termination logic in multiprocess manager #2371

Open abersheeran opened 6 days ago

abersheeran commented 6 days ago

Summary

About https://github.com/encode/uvicorn/discussions/2369

In our cluster, we accidentally discovered the zombie process. We found the reason. Uvicorn's new process manager will JOIN child processes one by one after sending all exit signals. When the previous child process does not exit for a long time, the subsequent child processes cannot be JOIN.

I noticed that Uvicorn has an inherent shutdown timeout, which would be nice if we could use it with a multiprocessor.

The reason why we don't use terminate&join sequentially is to kill all processes faster, as mentioned in this PR https://github.com/encode/uvicorn/pull/2010

Checklist

abersheeran commented 6 days ago

I will add unit tests later, but I have no experience in designing a process that will hang, so if anyone can help, it would be greatly appreciated.