Synching two cameras ?

[kfarivar]

I am thinking about using multiple cameras and I know I need to synchronize them. Is it possible to access multiple cameras with this wrapper and can I synch the cameras (as the kinect SDK docs says) by doing something like this :

k4a_device_configuration_t deviceConfig;
deviceConfig.wired_sync_mode = K4A_WIRED_SYNC_MODE_SUBORDINATE

with the wrapper ?

[lpasselin]

yes see the config file https://github.com/etiennedub/pyk4a/blob/master/pyk4a/config.py#L47 use here: https://github.com/etiennedub/pyk4a/blob/master/pyk4a/config.py#L82

I use the sync signal (kinect as master) to control a light system and it works.

I am not sure if we've tried pyk4a with multiple devices but it should work.

[shagren]

It works. As minimum 2 devices in master/subordinate mode worked perfect. You must to know:

• you need to connect devices with sync cable
• master must be started with depth+color mode, subordinates can start with depth-only mode
• all devices must has same firmware version
• subordinates must be started before master
• you need to configure subordinate_delay_off_master for each subordinate. You can use 160 usec for first one, 320 for second, 480 for third ...
• you can setup maximum 9 devices
• you need to read captures first from master then from subordinates

[kimtaehyeong]

Hello. I am also setting up the camera to synchronize. If possible, could you please post an example code sample?

Thank you.

[lpasselin]

Try it and share with us the code if it doesn't work?

[kimtaehyeong]

my code is

""" import cv2

import pyk4a from example.helpers import colorize from pyk4a import Config, PyK4A import numpy as np img_count = 1

def main():

global img_count
k4a_1 = PyK4A(Config(wired_sync_mode=1,camera_fps=0, color_resolution=pyk4a.ColorResolution.RES_720P, depth_mode=pyk4a.DepthMode.WFOV_UNBINNED,))
k4a_1.start()
k4a_2 = PyK4A(Config(wired_sync_mode=2,camera_fps=0, color_resolution=pyk4a.ColorResolution.RES_720P, depth_mode=pyk4a.DepthMode.WFOV_UNBINNED,))
k4a_2.start()

while True:
    capture = k4a_1.get_capture()
    capture_1 = k4a_2.get_capture()
    if capture.depth is not None:
        cv2.imshow("Depth", colorize(capture.depth, (None, 5000)))
    if capture.ir is not None:
        cv2.imshow("IR", colorize(capture.ir, (None, 1000), colormap=cv2.COLORMAP_JET))
    if capture.color is not None:
        cv2.imshow("Color", capture.color)
    # if capture.transformed_depth is not None:
    #     cv2.imshow("Transformed Depth", colorize(capture.transformed_depth, (None, 5000)))
    # if capture.transformed_color is not None:
    #     cv2.imshow("Transformed Color", capture.transformed_color)
    # if capture.transformed_ir is not None:
    #     cv2.imshow("Transformed IR", colorize(capture.transformed_ir, (None, 500), colormap=cv2.COLORMAP_JET))

    key = cv2.waitKey(10)
    if key == 27:
        cv2.destroyAllWindows()
        break
    if key == ord('s'):
        ir = capture.ir
        dp = capture.depth
        ir = ir / 4096 * 250
        #ir -= 1024
        ir[ir > 250] = 250
        ir[ir < 0] = 0
        dp = dp / 4096 * 250
        dp[dp > 250] = 250
        dp[dp < 0] = 0
        #cv2.imwrite('save_img/depth{}.jpg'.format(str(img_count)),dp)
        #cv2.imwrite('save_img/color{}.jpg'.format(str(img_count)),capture.color)
        #cv2.imwrite('save_img/ir{}.jpg'.format(str(img_count)),ir)
        #cv2.imwrite('save_img/transformed_ir{}.jpg'.format(str(img_count)),capture.transformed_ir)
        #cv2.imwrite('save_img/transformed_depth{}.jpg'.format(str(img_count)),capture.transformed_depth)
        cv2.imwrite('save_img/transformed_color{}.jpg'.format(str(img_count)),capture.transformed_color)

        cv2.imwrite('save_img/transformed_color_1{}.jpg'.format(str(img_count)),capture_1.transformed_ir)
        img_count = img_count + 1

k4a_1.stop()
k4a_2.stop()

if name == "main": main()

""" Thanks for your quick reply.

[shagren]

1. You need additional cable for connecting cameras with each other, please read https://docs.microsoft.com/en-US/azure/Kinect-dk/multi-camera-sync
2. You must use for master camera:

   Config(
     ...
     wired_sync_mode = WiredSyncMode.MASTER
     ...
   )

And for other camera(s):

   Config(
      ...
      wired_sync_mode = WiredSyncMode.SUBORDINATE,
      subordinate_delay_usec = 160, # 320, 480, ... for next subordinates 
      ...
   )

3. color_resolution, depth_mode and fps must be identical for all cameras configs
4. First read capture from master camera, then from other cameras.
5. Sometimes sync failed and captures can be from different phases due internal queue. There are some ideas but I think you can just ignore that situation: https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/04168adc949c44180c51d90a8b8d34807891ec85/examples/green_screen/MultiDeviceCapturer.h#L95

BTW, storing images to disk can be expensive operation, maybe you can keep some images in memory and save later.

[kimtaehyeong]

Thanks for the good answer.

If you connect the 3.5 mm terminal, the setting is finished. Could you please write a simple example code for the capture? Thank you very much.