Support ToF while media-controller is enabled

[pascalau]

Hi Arducam-Team,

I was wondering if it is possible to support the ToF camera while the media-controller API on the Raspberry Pi is enabled. I tried using the ToF camera (connected on CAM0) simultaneously with an Arducam OG02B10 (B0348) (connected on CAM1) on a CM4. Both camera use the pivarity driver and are correctly identified by the system. Kernel version is 6.1.21-v8+. Other software I installed are libcamera according to the Getting Started section and the ToF sdk according to the instructions in this repository. I tried the following three /boot/config.txt settings:

1

dtoverlay=arducam-pivariety,media-controller=1
dtoverlay=arducam-pivariety,cam0,media-controller=0

This did unfortunately not work it seemed that the media-controller=0 setting override the setting for the other camera. This interferes with the operation of the normal camera since libcamera relies on the camera being handled by the media-controller API. The ToF camera works as intended and is fully functional.

2

dtoverlay=arducam-pivariety,media-controller=0

Same scenario as above.

3

dtoverlay=arducam-pivariety,media-controller=1

In this configuration the camera (OG02B10) works as intended. The ToF camera also works partially. I can request and get valid images. This works for all my intents and purposes. The only thing failing is trying to configure the camera to either the 2m or 4m mode. When the corresponding command is issued tof.setControl(ControlID::RANGE, MAX_DISTANCE); the following error is raised:

VIDIOC_S_CTRL: Inappropriate ioctl for device

I assume this is caused due to the V4L2 API call being directed to the '/dev/video0' device instead of the corresponding parent /dev/v4l-subdev0. Would it be possible to add this functionality to the SDK? Please let me know if you need any additional informations from my side or if I can test something on my setup.

Best,

[dennis-ard]

Hello, This is a problem we have always wanted to solve, but we have not figured out the affiliation between subdev and video. It would be great if you could provide references. Or I can give you a version that specifies the subdev interface. Best,

[pascalau]

Hi Dennis,

Thank you for your response. Here are links to helpful references:

• Official media-controller API documentation
• STM Wiki entry on V4L2 (not same device but still nice write up)
• Sticky on RPI forum concerning media-controller API

Enabling the media-controller basically sets up the following topology:

media-ctl -p
Media controller API version 6.1.21

Media device information
------------------------
driver          unicam
model           unicam
serial          
bus info        platform:fe800000.csi
hw revision     0x0
driver version  6.1.21

Device topology
- entity 1: arducam-pivariety 0-000c (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev0
    pad0: Source
        [fmt:Y12_1X12/240x180 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range
         crop.bounds:(0,0)/1600x1300
         crop:(0,0)/1600x1300]
        -> "unicam-image":0 [ENABLED,IMMUTABLE]

- entity 3: unicam-image (1 pad, 1 link)
            type Node subtype V4L flags 1
            device node name /dev/video0
    pad0: Sink
        <- "arducam-pivariety 0-000c":0 [ENABLED,IMMUTABLE]

The first entity is the sensor while the second one is its device node for video. I assume that the setting to configure the range has to be issued to the sensor itself. Therefore while configuring this setting it should not be issued to (/dev/video0) but rather /dev/v4l-subdev0 (see here). I don't know if the pipeline has to be configured in any other way (framerate/resolution/...) but all these kind of settings would have to be set for all entities from source to sink.

[dennis-ard]

hi, pascalau Thank you so much for sharing those resources with me! I appreciate your help and generosity in sharing your knowledge with me.

Best regards,

[dennis-ard]

hi, https://github.com/ArduCAM/arducam_ppa/pull/18, Media controller is supported in this arducam_tof_sdk v0.1.0 update.

Best regards,

[pascalau]

Hi Dennis Thank you very much for your help. I updated to the new SDK and can confirm that no error is raised when changing from 2m to 4m mode. Although I have seen that at least the ros example is broken due to the new enum CameraCtrl. I will close this issue and hopefully make a PR if I come around on some compilation issues due to the new headers.

[pascalau]

Sorry I didn't update my local repository of this repo. Compilation of all examples succeeded.