Unexplained High Concurrency Handling in Gunicorn with Uvicorn Workers

[leanderloew]

Hello Gunicorn community, I'm encountering unexpected behavior with concurrency handling in Gunicorn when using Uvicorn workers and FastApi and would appreciate some guidance.

Environment: Gunicorn with Uvicorn workers FastAPI application framework

Issue Description: Created a simple FastAPI endpoint that logs, sleeps for 10 seconds, and logs again before responding.

# FastAPI app
from fastapi import FastAPI
from fastapi.responses import PlainTextResponse
import logging
import time

logger = logging.getLogger()
app = FastAPI()

@app.get("/test")
def run_stage() -> PlainTextResponse:
    logger.info("Running test")
    time.sleep(10)
    logger.info("Finished test")
    return PlainTextResponse("Done!")

# Concurrency testing script
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import time

def make_requests(num_requests, url):
    def make_request():
        response = requests.get(url)
        return response.elapsed.total_seconds()

    with ThreadPoolExecutor(max_workers=num_requests) as executor:
        futures = [executor.submit(make_request) for _ in range(num_requests)]
        times = [future.result() for future in as_completed(futures)]

    return times

Expected vs Actual Behavior: Expected: The number of concurrent requests handled would be limited to the number of Gunicorn workers configured. Actual: A single Uvicorn worker handles 40 concurrent requests, scaling up from there without a clear rule.

Gunicorn Configuration: gunicorn -k uvicorn.workers.UvicornWorker -w $UVI_WORKERS -t $TIMEOUT --bind "0.0.0.0:$PORT" --preload "path_to_my_app:app"

Attempted Fixes: Reviewed existing documentation and searched for similar cases without finding a clear explanation for this behavior.

Conclusion: I am looking for documentation or insights into how many concurrent requests Gunicorn with Uvicorn workers is expected to handle and how this number is derived / scales with UVI workers.

[riwatt]

Expected: The number of concurrent requests handled would be limited to the number of Gunicorn workers configured.

This is expected to hold true only for sync workers.

I am looking for documentation or insights into how many concurrent requests Gunicorn with Uvicorn workers is expected to handle and how this number is derived / scales with UVI workers.

See https://docs.gunicorn.org/en/latest/design.html

Gunicorn creates worker processes and distributes requests among them. It's up to each worker's implementation and configuration to decide its own concurrency. In the above mentioned (and default) sync worker's case it's a single request per worker process.

Uvicorn workers by default don't limit their concurrency but it's possible with some customization. There is some discussion here on the relevant settings.