search

mrhan1993 / Fooocus-API

FastAPI powered API for Fooocus
GNU General Public License v3.0
558 stars 148 forks source link

Task Queue Not Working Properly When There Are Multiple Instances #92

Closed thiner closed 8 months ago

thiner commented 8 months ago

I deployed Fooocus-API to k8s cluster, and started two pods to serve client request. When client enable async request, the query job status request may fail and get a 404 response. This is caused by the job queue design. The job queue is not centralized, so the job id is not unique among all instances.

Untitled Diagram-Page-8 drawio

mrhan1993 commented 8 months ago

Fooocus api is not currently designed for clusters, so what you say happens when you schedule to another Pod, you can try using service's LeastConnection load balancing method, or Session Affinity to ensure that requests are always routed to the correct Pod