Memory leak when running Flask app with multithreading background processes

[hoang-innomize]

Please provide us with the following information:

This issue is a: (mark with an x)

• [ x] bug report -> please search issues before submitting
• [ ] documentation issue or request
• [ ] regression (a behavior that used to work and stopped in a new release)

Issue description

I am deploying a Flask app on Azure Container App consumption mode using 3 vCPUs and 6 GB RAM. This is a long-running Flask API and to avoid complex in the design, we have some APIs that we will process requests in the background using multithreading. After receiving a request, we start a new thread and return response to the end user. We have noticed an issue that when the threads process the background job, the memory usage is increasing and it keep there without cleaning up.

Expected behavior

After finishing the background job, basically the memory should be free up and reduce the memory usage.

Actual behavior

Memory keeps raising without releasing after processing the job.

Additional context

Dockerfile to run the API

# Use an official Python runtime as a parent image
FROM python:3.10-slim-buster

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Create and set the working directory in the container
WORKDIR /app

# Install libs for opencv
RUN apt -y update && apt -y upgrade
RUN apt -y install libopencv-dev

# Copy the requirements file into the container at /app
COPY requirements.txt /app/

# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of the application code into the container
COPY . /app/

# Expose a port that your Flask app will run on (e.g., 5000)
EXPOSE 5000

ENV FLASK_ENV production

RUN pip install gunicorn[gthread]

# Run your Flask app with WSGI Http Server
CMD ["gunicorn", "--bind", "0.0.0.0:5000", "--access-logfile", "-", "--error-logfile", "-", "-w", "4", "--threads", "3", "run:flask_app"]

Sample code to start threads

def run_job(user_id, items):
    created_items = create_items(user_id, items)

    Thread(target=start_background_job, args=[user_id, created_items]).start()

    return created_items

Python packages

Flask==3.0.0 Flask-Caching==2.3.0 flask-restx==1.3.0 gunicorn==22.0.0

[anthonychu]

Do you get the same problem when you run your container locally?

[microsoft-github-policy-service[bot]]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.