How to acquire synchronised frames from 2 Hardware Triggered Cameras- especially synchronised start and end?

[aruna-ram]

Hi,

I am having trouble understanding how to acquire images taken at the same point in time from two cameras. I assume images are taken at the same time since both cameras are told to 'FrameStart' upon receiving signal from the same hardware trigger. However when I retrieve the images and retrieve the time stamp from the images, the stamps do not seem to be synchronised (as plotted below) as one would expect. Importantly: the first camera starts and ends before the second camera. I suspect that this might have something to do with my grabbing strategy loop.

Below is the code that I am using from this pypylon sample notebook (mutlicamera handling). My only addition was to extract and save the images and the timestamps associated with them. I am using a 100Hz signal from a Raspberry Pi.

What is the best way to have two hardware triggered cameras and extract synchronised frames from them (i.e. the first image grabbed, captured by camera 0, was taken at the same time as the first image grabbed from camera 1)?

"""
Records images from 2 hardware triggered cameras
"""
import pypylon.pylon as py 
import time
import matplotlib.pyplot as plt
import numpy as np 

NUM_CAMERAS = 2

tlf = py.TlFactory.GetInstance()
di = py.DeviceInfo()
#di.SetDeviceClass("BaslerCamEmu")
#device list
devs = tlf.EnumerateDevices([di,])

print(devs)

#control features of multiple cameras at once
cam_array = py.InstantCameraArray(NUM_CAMERAS)

#attaches index 
for idx, cam in enumerate(cam_array):
    cam.Attach(tlf.CreateDevice(devs[idx]))

cam_array.Open()

# store a unique number for each camera to identify the incoming images
for idx, cam in enumerate(cam_array):
    camera_serial = cam.DeviceInfo.GetSerialNumber()
    print(f"set context {idx} for camera {camera_serial}")
    cam.SetCameraContext(idx)

#set hardware trigger 
for camera in cam_array:
    camera.LineSelector = "Line4"
    camera.LineMode = "Input"
    camera.TriggerSelector = "FrameStart"
    camera.TriggerSource = "Line4"
    camera.TriggerMode = "On"
    camera.TriggerActivation.Value

img_list_0= []
img_list_1 = []
time_list_0 = []
time_list_1 = []
# wait for all cameras to grab 10 frames
frames_to_grab = 10
# store last framecount in array
frame_counts = [0]*NUM_CAMERAS

cam_array.StartGrabbing()
while True:
    with cam_array.RetrieveResult(1000) as res:
        if res.GrabSucceeded():
            img_nr = res.ImageNumber
            cam_id = res.GetCameraContext()
            frame_counts[cam_id] = img_nr
            print(f"cam #{cam_id}  image #{img_nr}")
            img = res.GetArray()
            time_stamp = res.TimeStamp
            res.Release()
            if cam_id == 0:
                img_list_0.append(img)
                time_list_0.append(time_stamp)
            else:
                img_list_1.append(img)
                time_list_1.append(time_stamp)

            # check if all cameras have reached 100 images
            if min(frame_counts) >= frames_to_grab:
                print(f"all cameras have acquired {frames_to_grab} frames")
                break

cam_array.StopGrabbing()
cam_array.Close()

[thiesmoeller]

The timestamps on Basler cameras have a resolution around 10ns per tick. Even if you have two cameras with synchronized clocks and both are triggered from the same signal small differences in the electrical circuits and your cable lengths will result in different timestamps.

Long story short: it is impossible to compare timestamps on equality.

The standard way to operate multiple cameras with common HW trigger is:

The trigger signal is idle. You call StartStreaming for all your cameras. Start triggering. Now you can Retrieve the images. With each trigger you get a set of images from the cameras.

With this procedure the first trigger will result in first image from all cameras, second trigger will result in second image from all cameras. So you can match your images on the frameID in each frame.

[aruna-ram]

Hi Thies, thanks for your answer! This is useful to know. I haven't seen the StartStreaming method in PyPylon before, would you be able to point me to any example/documentation? I was also starting my trigger before I ran cameras.StartGrabbing() but I see why this would be an issue if this command reached the cameras at different times. I'll follow this method and see how it goes.

[thiesmoeller]

Ups ;-) I meant StartGrabbing of course. So everything is right what you do, just keep trigger silent before all cameras are started

If available on your camera also recommended to use the trigger input counter chunk https://docs.baslerweb.com/data-chunks#trigger-input-counter-chunk This allows to always check, that the frameID is in sync with detected triggers on the camera

[adahbingee]

Finally works. Great tips!!! keep trigger silent before all cameras are started