[Bug]: compute_face_descriptors is not releasing memory properly

[MahmoudMoumni]

What Operating System(s) are you seeing this problem on?

Linux (aarch64)

dlib version

19.21.0

Python version

3.6.9

Compiler

gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0

Expected Behavior

The function compute_face_descriptors should release memory after execution,but i's not the case

Current Behavior

compute_face_descriptors is not releasing memory when being used in another thread

Steps to Reproduce

import os
import cv2
import numpy as np
import pickle
import mediapipe as mp
import dlib
import face_recognition
import face_recognition_models
import psutil
from concurrent.futures import ThreadPoolExecutor
import gc

def print_ram_usage(stage):
    pid = os.getpid()
    process = psutil.Process(pid)
    ram_usage = process.memory_info().rss / 1024 ** 2  # Convert bytes to MB
    print(f"{pid} {stage}: {ram_usage} MB")

def compute_face_descriptor_async(rgb_frame, raw_landmarks, num_jitters=1):
    face_recognition_model = face_recognition_models.face_recognition_model_location()
    face_encoder = dlib.face_recognition_model_v1(face_recognition_model)
    rt = face_encoder.compute_face_descriptor(rgb_frame, raw_landmarks, num_jitters=num_jitters)
    del face_recognition_model
    del face_encoder
    gc.collect()
    return rt

# Initialize Mediapipe Face Mesh and Drawing modules
mp_face_mesh = mp.solutions.face_mesh
mp_drawing = mp.solutions.drawing_utils

# Initialize the Face Mesh model
face_mesh = mp_face_mesh.FaceMesh(
    static_image_mode=False,
    max_num_faces=1,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5
)

# Define drawing specifications
drawing_spec = mp_drawing.DrawingSpec(color=(0, 255, 0), thickness=1, circle_radius=1)

frame = cv2.imread("../face_captures/tilt_0.jpg")
# Flip the frame (e.g., horizontally)
frame = cv2.flip(frame, 1)

# Convert the frame to RGB
rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
padding = 10
# Detect faces in the frame
face_locations = []
# Process the frame and detect face mesh
results = face_mesh.process(rgb_frame)
# Draw the face mesh annotations and bounding box on the frame
if results.multi_face_landmarks:
    for face_landmarks in results.multi_face_landmarks:
        # Get the landmark coordinates
        h, w, _ = frame.shape
        landmark_coords = [(int(point.x * w), int(point.y * h)) for point in face_landmarks.landmark]

        # Calculate the bounding box coordinates with padding
        x_min = max(min([coord[0] for coord in landmark_coords]) - int(padding / 2), 0)
        y_min = max(min([coord[1] for coord in landmark_coords]) - padding, 0)
        x_max = min(max([coord[0] for coord in landmark_coords]) + int(padding / 2), w)
        y_max = min(max([coord[1] for coord in landmark_coords]) + padding, h)

        # Append coordinates to face_locations in the correct format
        face_locations.append((y_min, x_max, y_max, x_min))

        # Draw the bounding box
        cv2.rectangle(frame, (x_min, y_min), (x_max, y_max), (0, 255, 0), 2)

print_ram_usage(0)
raw_landmarks = face_recognition.api._raw_face_landmarks(rgb_frame, face_locations)
print_ram_usage(1)

# Using ThreadPoolExecutor to execute compute_face_descriptor asynchronously
with ThreadPoolExecutor() as executor:
    future = executor.submit(compute_face_descriptor_async, rgb_frame, raw_landmarks[0])
    rt = future.result()

# Clean up resources used by the thread
del future
executor.shutdown(wait=True)
gc.collect()

print_ram_usage(2)

# Clear large objects
face_locations.clear()
results = None
gc.collect()

print_ram_usage(3)

video_capture = cv2.VideoCapture(0)
while True:
    # Capture frame-by-frame
    ret, frame = video_capture.read()
    if not ret:
        print("Error: Could not read frame.")
        break
    # Display the resulting frame
    cv2.imshow('Video', frame)

    # Break the loop on 'q' key press
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

print_ram_usage(4)
# Release the capture, the VideoWriter, and close any OpenCV windows
video_capture.release()
cv2.destroyAllWindows()
gc.collect()

print_ram_usage(5)

Anything else?

No response

[davisking]

I don't know what's inside the face_recognition. Please post a minimal example that uses just dlib and shows a memory leak. I strongly expect there isn't a leak in the dlib part of this :)

[MahmoudMoumni]

Dear Mr Davis

Please check this link for the issue https://github.com/davisking/dlib/issues/2960 You can see that the face_encoder is an instance of dlib.face_recognition_model_v1 and im using the compute_face_descriptor from dlib.face_recognition_model_v1

I will send you another example not including mediapipe so you can run it easily.

Thanks for’ replying

Sincerely Mahmoud

On Sat, 1 Jun 2024 at 14:02, Davis E. King @.***> wrote:

I don't know what's inside the face_recognition. Please post a minimal example that uses just dlib and shows a memory leak. I strongly expect there isn't a leak in the dlib part of this :)

— Reply to this email directly, view it on GitHub https://github.com/davisking/dlib/issues/2960#issuecomment-2143423817, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJDEJUZTLJP5IKNXHMU5XXLZFGZ7DAVCNFSM6AAAAABIPNXDGGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGQZDGOBRG4 . You are receiving this because you authored the thread.Message ID: @.***>

[davisking]

Depending on what all is happening, you may also be noticing that the cuda runtime is holding onto memory. There is some amount of that that is uncontrollable. It's not a leak and not something to worry about.

[MahmoudMoumni]

Dear Mr David,

The amount of memory that’s not released is almost 1GB , and im working on a software that uses face authentication, this memory won’t be released after i finish the face authentication process and since im working on jetson nano (4GB ram ) it will be very important to just load and use authentication and then release memory for the rest of pipeline , while this is not happening!

I will create a very simple script and print the ram at different stages and send it to you.

I think its very important to release memory when the scope finishes and im trying my best with source code to check which part is holding that memory !

I will get back to you soon

On Sat, 1 Jun 2024 at 14:28, Davis E. King @.***> wrote:

Depending on what all is happening, you may also be noticing that the cuda runtime is holding onto memory. There is some amount of that that is uncontrollable. It's not a leak and not something to worry about.

— Reply to this email directly, view it on GitHub https://github.com/davisking/dlib/issues/2960#issuecomment-2143431539, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJDEJUYCPWVBOB3QSPIM5GDZFG47PAVCNFSM6AAAAABIPNXDGGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGQZTCNJTHE . You are receiving this because you authored the thread.Message ID: @.***>

[dlib-issue-bot]

Warning: this issue has been inactive for 35 days and will be automatically closed on 2024-07-16 if there is no further activity.

If you are waiting for a response but haven't received one it's possible your question is somehow inappropriate. E.g. it is off topic, you didn't follow the issue submission instructions, or your question is easily answerable by reading the FAQ, dlib's official compilation instructions, dlib's API documentation, or a Google search.

[dlib-issue-bot]

Warning: this issue has been inactive for 42 days and will be automatically closed on 2024-07-16 if there is no further activity.

If you are waiting for a response but haven't received one it's possible your question is somehow inappropriate. E.g. it is off topic, you didn't follow the issue submission instructions, or your question is easily answerable by reading the FAQ, dlib's official compilation instructions, dlib's API documentation, or a Google search.

[dlib-issue-bot]

Notice: this issue has been closed because it has been inactive for 45 days. You may reopen this issue if it has been closed in error.