Docker container FastAPI backend is not streaming, but waiting to send full payload

This issue is for a: (mark with an `x`)

[x] bug report
[ ] feature request
[ ] documentation issue or request
[ ] regression (a behavior that used to work and stopped in a new release)

Description We have deployed a docker container with a FastAPI backend, that uses Azure OpenAI service to generate streaming responses. It's used as a part of a larger setup with a frontend to enable users to interact with OpenAI LLM's. When running to FastAPI locally it works as intended and streams the response, but when deploying to Azure Web App it does not.

Minimal steps to reproduce

1) Setup Azure Web App 2) Setup simple FastAPI app with a streamingresponse endpoint 3) Deploy simple FastAPI app in docker container to Azure Web App 4) Send a request to endpoint

Any log messages given by the failure

Logs seem to behave as intended, it looks like it's actually performing the streaming steps, however no payload is returned until it finishes.

Expected/desired behavior

Expects a streaming response, but waits for the entire payload. Screenshots Examples of local behavior vs Azure web app: Local streaming, indicating the number of chunks/elements recieved:

Now using the same request, but sending to the Azure Web App(contains multiple elements, but in one chunk):

Select Web App settings:

"kind": "app,linux,container",
    "siteConfig": {
        "numberOfWorkers": 1,
         "linuxFxVersion": "DOCKER|<REPO>/<IMAGE>",,
        "acrUseManagedIdentityCreds": false,
        "alwaysOn": true,
        "http20Enabled": true,
        "functionAppScaleLimit": 10,
        "minimumElasticInstanceCount": 2
    },

Docker image:

FROM python:3.10-alpine3.19

WORKDIR /app

RUN pip install -U pip setuptools wheel
RUN pip install poetry

COPY ./ /
RUN poetry install --no-dev
ENTRYPOINT ["poetry","run","gunicorn","-w","4","-k","uvicorn.workers.UvicornWorker","--timeout","600","app:app"]

Poetry dependencies:

python = "^3.10"
pytest = "^7.4.3"
azure-identity = "^1.15.0"
openai = "0.28"
asynctest = "^0.13.0"
uvicorn = "^0.24.0.post1"
fastapi = "0.110.1"
httpx = "^0.25.1"
fastapi-azure-auth = "^4.3.0"
pydantic = "^2.5.2"
pydantic-settings = "^2.1.0"
nanoid = "^2.0.0"
tiktoken = "^0.5.1"
python-dotenv = "^1.0.0"
pytest-cov = "^4.1.0"
pytest-azurepipelines = "^1.0.5"
flake8 = "^6.1.0"
autopep8 = "^2.0.4"
azure-cosmos = "^4.5.1"
azure-keyvault-secrets = "^4.7.0"
applicationinsights = "^0.11.10"
pytest-rerunfailures = "^13.0"
cryptography = "42.0.4"
aiohttp = "3.9.4"
idna = "3.7"
gunicorn = "^22.0.0"

Thanks! We'll be in touch soon.

Azure / app-service-linux-docs