Issue with Multiprocessing and Camera Connection Using Basler a2A4504-18ucBAS

[diezm]

Describe what you want to implement and what the issue & the steps to reproduce it are:

I'm developing a Flask web application that incorporates object detection. The application utilizes the multiprocessing module to handle camera frame capture and image processing in parallel. Specifically, one process is dedicated to reading frames from a Basler camera and enqueueing them into a Queue, while another process dequeues these frames for further processing.

This setup worked seamlessly with a Basler a2A2448-105g5cBAS (Ethernet) camera. However, upon switching to a newer model, Basler a2A4504-18ucBAS (USB3.0), the application no longer functions as expected.

The core of the application relies on a singleton Basler class, ensuring a single camera instance throughout the application's lifecycle. This approach was effective with the previous camera model.

The issue arises when the main process attempts to connect to the new camera to fetch initial parameters (e.g., exposure time, pixel format) before launching the frame capture subprocess. With the new camera model, the subprocess seems unable to establish a connection to the camera, resulting in no frames being captured or processed.

Strikingly, if I bypass the initial parameter fetching in the main process (as indicated in the code block below), the frame capture subprocess operates without issue, suggesting a conflict when connecting to the camera across multiple processes.

However, accessing camera parameters before starting the frame capture process is crucial for my application's functionality.

I'm seeking insights or solutions to address this multiprocessing and camera connection issue with the Basler a2A4504-18ucBAS model. Any advice or workaround to maintain the initial parameter fetching while ensuring seamless frame capture in the subprocess would be greatly appreciated.

from multiprocessing import Process, Value
import time
from pypylon import pylon

class Basler:
    _instance = None

    def __new__(cls):
        if cls._instance is None:
            cls._instance = super(Basler, cls).__new__(cls)
            cls._instance.camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())
            cls._instance.is_streaming = Value('b', False)
        return cls._instance

    def connect(self):
        if not self.camera.IsOpen():
            self.camera.Open()
            print('Camera initialized')
        else:
            print('Camera already initialized')

    def start_streaming(self):
        self.camera.StartGrabbing(pylon.GrabStrategy_LatestImageOnly)
        self.is_streaming.value = True
        print('Streaming started')

    def get_frame(self):
        if self.camera.IsGrabbing():
            grab_result = self.camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)
            if grab_result.GrabSucceeded():
                return grab_result.Array

    def stop_streaming(self):
        self.camera.StopGrabbing()
        self.is_streaming.value = False
        print('Streaming stopped')

    def disconnect(self):
        self.camera.Close()
        print('Camera disconnected')

def camera_process(start_process):
    cam = Basler()
    cam.connect()
    cam.start_streaming()
    while start_process.value:
        frame = cam.get_frame()
        if frame is not None:
            print("Frame captured")
    cam.stop_streaming()
    cam.disconnect()

if __name__ == "__main__":

    #######################################################################################
    #######################################################################################
    ############### Without this block the code works for me ###############################
    cam = Basler()
    cam.connect()
    time.sleep(.2)
    cam.disconnect()
    #######################################################################################
    #######################################################################################
    #######################################################################################

    start_process = Value('b', True)
    camera_proc = Process(target=camera_process, args=(start_process,))
    camera_proc.start()

    # Run the camera process for a short duration then stop
    time.sleep(2)  # Adjust duration as needed
    start_process.value = False

    camera_proc.join()

    print("Script ended.")

Is your camera operational in Basler pylon viewer on your platform

Yes

Hardware setup & camera model(s) used

Ubuntu 22.04.3 LTS Python 3.11.0

Cameras: a2A2448-105g5cBAS (Ethernet), no problems a2A4504-18ucBAS (USB3.0), problem as described above

Runtime information:

python: 3.11.0 (main, Mar  1 2023, 18:26:19) [GCC 11.2.0]
platform: linux/x86_64/6.2.0-36-generic
pypylon: 1.9.0 / 6.3.0.18933

[thiesmoeller]

your code will try to transfer the InstantCamera between processes.

 def __new__(cls):
        if cls._instance is None:
            cls._instance = super(Basler, cls).__new__(cls)
            cls._instance.camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())
            cls._instance.is_streaming = Value('b', False)
        return cls._instance

The communication in python multiprocessing is via pickling/unpickling of objects between the processes. This doesn't work and would be extremely inefficient for the InstantCamera.

Either create different InstantCameras in each process ( although only one have to have the camera "open" at any time ) Or by adding additional methods to set/get parameters or a list of parameters from the other process. The objects transferred have to be pure python.

In https://github.com/basler/pypylon/issues/513#issuecomment-1346405311 I gave an example how to run pypylon in one process and transfer video data to other process via shared memory to be efficient.

[diezm]

Thanks for your reply. I

Either create different InstantCameras in each process ( although only one have to have the camera "open" at any time )

I tried this, but it also did not work.

The problem described above only occurs when the program structure is like this:

Main Process
.... do some stuff ....
open camera
read params
close camera
       start subprocess
       open camera

I found a workaround by starting a subprocess just to read out the camera params now, this works

Main Process
.... do some stuff ....

      start subprocess_1
      open camera
      read params
      close cam
      kill subprocess

      start subprocess_2
      open camera
      start streaming 

[jonahpearl]

We had the same issue -- your fix resolved it for us. Would still like to understand the root cause so we can avoid it!

EDIT: it appears to be an issue with multiprocessing using spawn instead of fork on Linux. The problem wasn't present on Windows, and we solved it equally well with:

current_mp_start_method = mp.get_start_method()
if current_mp_start_method != "spawn":
    mp.set_start_method("spawn", force=True)

[thiesmoeller]

The root cause is described in https://github.com/basler/pypylon/issues/714#issuecomment-1919397433.

You have to centralize the camera access ( InstantCamera) in one process. The object can't move between processes.

Best practice is linked in the above comment. Only process the data in sub processes. This can be optimized by using shared memory for the image data

[jonahpearl]

Ok — I think I'm experiencing a slightly distinct problem that is better described in #659 — going to take this over there.