search

basler / pypylon

The official python wrapper for the pylon Camera Software Suite
http://www.baslerweb.com
BSD 3-Clause "New" or "Revised" License
570 stars 207 forks source link

Synchronous operation of multiple cameras #590

Open Omeon opened 1 year ago

Omeon commented 1 year ago

Good day to everyone. I am writing here for help. We are creating a project that uses 8 Basler ace acA4112-20um cameras. The cameras are positioned in such a way that they continuously cover a certain area of 12 meters. And now we face the following problem: there are zones in the field of view of cameras, the sizes of which change from time to time. In each zone there is an object that is recognized and then, depending on the detection zone, it is entered into the database. The problem is how to handle the changing sizes of the zones, since today the zone deals with the area of shooting one camera, and tomorrow 3 cameras, the day after tomorrow already 0.5 of the area of 1 camera. It was proposed to conduct synchronous shooting from all cameras, and then glue the frames into one. This frame would be processed easily, I know the ratio of the area to the resolution of the camera. But our cameras do not support the PTP protocol, so we cannot use Basler's standard tools. Using HW launch will give synchronization only at the beginning of shooting, after some time it will disappear, given that the task is to process 25 fps. I would like to hear suggestions on how to resolve this issue. Thank you in advance.

thiesmoeller commented 1 year ago

What do you mean by HW launch? How do you handle the long distance of 12/6m with USB? Active cables?

The principle solution to operate multiple cameras in synch is to use triggered acquisition. On USB you have to wire up a HW trigger. This trigger can control the exact moment of acquisition.

To your setup: do you know the position of zone, where you will detect the image in advance ?

Omeon commented 1 year ago

Greetings, @thiesmoeller, thank you for your reply. By HW launch, I mean the use of I/O lines present in the cameras and which you probably talk about below.

Using the HW trigger will allow you to synchronize frames at a speed of 25 fps? Or will it allow you to synchronize only the launch of cameras at the very beginning? And what will be the speed of the camera's response to the received signal?

Yes, to connect the camera over long distances, we have purchased active Usb extension cords, which allows you to connect them at a distance of up to 30m. At the same time, there are no delays.

The size and location of the zones are known in advance before turning on the cameras.

thiesmoeller commented 1 year ago

If you inject the same 25Hz signal into all cameras I/O interface, due to the long cable lengths I would strongly recommend to use the optocoupled inputs and configure this as the FrameStart trigger, you get perfectly synchronized frames from all cameras. Frame by frame.

Now you can configure which camera will send the images and which portion of it. This can then be dynamically adjusted to your new product/production setup.

To identify which images belong to the same trigger, you can activate the trigger counter chunk. For this the external trigger generator has to be disabled before you start the image capturing. If every camera is ready you start the generator. Now all counters will run in sync.

Stitching the images directly might be tricky as this only works pixel exact, if your camera images are rectified and the differences in pose is corrected. ( Intrinsic/extrinsic calibration )

But if your requirements on precision are lower, you can choose other means of aligning your data.

Overall it helps to have overlap of the field of view of the cameras and marker/fiducials in this overlap area to measure misalignment of the cameras over this long 12m work area.

How do you connect all cameras to the PC? Multiple USB controllers? As 25fps from a single camera is already most of the total USB3 bandwidth in full resolution.

Omeon commented 1 year ago

Good time of day, @thiesmoeller.

Thanks for the advice about optocoupler inputs, I plan to use them.

Speaking of other ways to reconcile your data, what did you mean? I would love to hear your options.

Our accuracy does not involve pixel-to-pixel alignment, as the cameras are configured to slightly overlap the image to ensure a continuous flow of information.

The cameras are connected to the computer via a USB controller that is inserted into the motherboard. USB 3.0 bandwidth with a margin is enough to handle even all 8 cameras on one port, but just in case they installed 1 camera - 1 port. The data transfer rate from one camera is about 300 MB/s.