imdecode jpeg2000 segfault or deadlock in multithreading

[robinechuca]

Expected behaviour

In the main thread, it works well as written in the example bellow:

import multiprocessing.pool
import cv2
import numpy as np
import tqdm

img = np.random.randint(0, 65335, (1024, 1024), dtype=np.uint16)
_, img_data = cv2.imencode(".jp2", img)
n = 100

def read(i):
    image = cv2.imdecode(buffer, cv2.IMREAD_ANYDEPTH | cv2.IMREAD_GRAYSCALE)

list(tqdm.tqdm(map(read, range(n)), total=n, desc="main thread main process"))
with multiprocessing.pool.Pool() as pool:
   list(tqdm.tqdm(pool.imap_unordered(read, range(n)), total=n, desc="main thread child process"))

Actual behaviour

But in a thread, it is all broken!

import multiprocessing.pool
import cv2
import numpy as np
import tqdm

img = np.random.randint(0, 65335, (1024, 1024), dtype=np.uint16)
_, img_data = cv2.imencode(".jp2", img)
n = 100

def read(i):
    image = cv2.imdecode(buffer, cv2.IMREAD_ANYDEPTH | cv2.IMREAD_GRAYSCALE)

with multiprocessing.pool.ThreadPool() as pool:
   list(tqdm.tqdm(pool.imap_unordered(read, range(n)), total=n, desc="child thread main process"))

When the code does not freeze by itself, an interrupt with ctrl+c leads allways to a segfault. Unlike the threadless example.

Steps to reproduce

• Tested on Debian, Ubuntu and Mint
• Version 4.9.0 of opencv-contrib-python-headless
• Tested on python 3.9, 3.10, 3.11 and 3.12
• architecture x86

[opencv-alalek]

But in a thread

Ensure that there are real threads. Because forked processes don't work: https://github.com/opencv/opencv/issues/5150

[robinechuca]

I've just checked carefully. Whether with a context fork or spawn, the function executes perfectly. However, as soon as the thread, not the process, is no longer the principal, there seems to be a conflict with the GIL.

If we start a spawn process in each sub-threads. It works. But it is extremely slow to create a new spawn process each time! But to create a fork process in each sub-thread doesn't solve anything.