MIPI i2c device found but no dev /dev/video interface on IMX8M NXP port

[bitslip6]

Hello!

Thanks for your great help and feed back on getting our device tree running for the csi camera interface. Huge help.

So currently we have a device tree file that appears to be working for the 3dtof dev kit and the addi9036 kernel module ported to the 4.14.98 NXP linux kernel. The relevant output can be seen in the attached output, however I am not seeing a new /dev/video interface being created when I load the kernel module.

Keep in mind we are running a stock 4.14.98 Linux kernel provided by variscite and NXP.

Any help in what we need to do to get the /dev/videoX device created would be most helpful!

Thank you kindly in advance!! best regards, Cory

root@imx8mm-var-dart:~# modprobe addi9036
root@imx8mm-var-dart:~# dmesg
....
[ 1238.028836] addi9036 1-0064: addi9036_probe: i2c addr = 0x64
[ 1238.044749] addi9036 1-0064: addi9036 detected at address 0x64
[ 1238.058181] addi9036 1-0064: set_fmt: 0 640x960
[ 1238.061422] CSI: Registered sensor subdevice: addi9036 1-0064

root@imx8mm-var-dart:~# i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- UU -- -- -- -- 3d -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- 54 55 -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- UU -- -- -- UU -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                         

^^^^ notice the kernel recognizes the mipi i2c interface on i2c bus 1 (i2c2 in the schematics)

[btogorean]

We would like to also thank you for your great work in adding support on I.MX8 for ADI ToF sensor Cory.

I'm thinking that can be a problem related to the endpoint that ADI ToF is connected to being disabled. It is by default disabled in source devicetree include file fsl-imx8mq.dtsi. My approach to don't get lost on devicetree include tree is to decompile the .dtb file that is loaded at boot and have a look there.

Also you can access and browse the loaded devicetree using /proc/device-tree.

Also I suppose that output of v4l2-ctl --list-devices is null.

Regards, Bogdan

[bitslip6]

root@imx8mm-var-dart:# cat /proc/device-tree/csi1_bridge@32e20000
okay
root@imx8mm-var-dart:# v4l2-ctl --list-devices
i.MX6S_CSI (platform:32e20000.csi1_bridge):
    /dev/video0

csi1_bridge is defined in the dts as:

+   /* addi 9036 3d tof camera */
+   camera_front: camera_front@64 {
+       status = "okay";
+       compatible = "adi,addi9036";
+       reg = <0x64>;
+       reset-gpios = <&gpio5 28 GPIO_ACTIVE_HIGH>;
+
+       port {
+           addi9036_ep: endpoint {
+               remote-endpoint = <&csi1_ep>;
+               clock-lanes = <0>;
+               data-lanes = <1 2>;
+           };
+       };
+   };
+
+
+
 &csi1_bridge {
    fsl,mipi-mode;
    status = "okay";
+
    port {
        csi1_ep: endpoint {
-           remote-endpoint = <&csi1_mipi_ep>;
+           remote-endpoint = <&addi9036_ep>;
        };
    };
 };

So perhaps it is working after all, I need to reload the SDK today on the target and verify. I was expecting a new video device since /dev/video0 is always there. I will verify and update the ticket later today.

The raspberry-pi device tree overlay did not include settings for clock rates. Please advice clock rate settings, or if the AFE uses it's own oscillator.

[bitslip6]

I think I am missing a key component in this build. The i2c lines are functioning fine and the kernel sees the devices and is managing 0x56,0x57,0x64, however the mipi csi device doesn't seem to be functioning and I'm not sure how to debug the csi interface at this point.

any advice would be appreciated!

root@imx8mm-var-dart:~/aditof_sdk/build/examples/first-frame# v4l2-ctl --list-devices
i.MX6S_CSI (platform:32e20000.csi1_bridge):
    /dev/video0

root@imx8mm-var-dart:~/aditof_sdk/build/examples/first-frame# cat /proc/device-tree/csi1_bridge\@32e20000/status 
okay
root@imx8mm-var-dart:~/aditof_sdk/build/examples/first-frame#

root@imx8mm-var-dart:~/aditof_sdk/build/examples/first-frame# ls -l /dev/video* 
crw-rw---- 1 root video 81, 0 May  1 08:18 /dev/video0

root@imx8mm-var-dart:~/aditof_sdk/build/examples/first-frame# ./first-frame 
I0501 08:40:14.760049  1104 device_enumerator_raspberry.cpp:47] Looking for devices on the target
I0501 08:40:14.761103  1104 device_enumerator_raspberry.cpp:80] Looking at: /dev/video0 for an eligible TOF camera
I0501 08:40:14.762740  1104 system_impl.cpp:62] System initialized
I0501 08:40:14.762845  1104 camera_96tof1.cpp:78] Initializing camera
I0501 08:40:14.762866  1104 local_device.cpp:158] Opening device
W0501 08:40:14.773968  1104 local_device.cpp:198] CAPTURE Device i.MX6S_CSI
W0501 08:40:14.774204  1104 camera_96tof1.cpp:83] Failed to open device
E0501 08:40:14.774230  1104 main.cpp:64] Could not initialize camera!
W0501 08:40:14.783756  1104 local_device.cpp:147] close m_implData->sfd error errno: 9 error: Bad file descriptor
Segmentation fault (core dumped)

[btogorean]

Good to hear that /dev/video0 is up.

Unfortunately because of the FW from EEPROM is loaded only by SDK we can't start the video0 with tools like QT V4L2 to test the frame acquisition. So we have to tweak the also the SDK to adapt it a bit for I.MX8.

We have here a check for the right capture device by name. That name is hard-coded in target_definitions.h for each platform we support.

In I.MX8 case it looks like the capture device name read from video0 is CAPTURE Device i.MX6S_CSI but if you are using the raspberrypi buid SW requires unicam https://github.com/analogdevicesinc/aditof_sdk/blob/master/sdk/src/target/raspberrypi/target_definitions.h#L46

So please update that accordingly and check again.

[bitslip6]

Thanks much for this detail. I have updated target definition and made it a bit further.

root@imx8mm-var-dart:~/aditof_sdk/build/examples/first-frame# ./first-frame 
I0505 20:33:48.064162   609 device_enumerator_raspberry.cpp:47] Looking for devices on the target
I0505 20:33:48.065878   609 device_enumerator_raspberry.cpp:80] Looking at: /dev/video0 for an eligible TOF camera
I0505 20:33:48.067306   609 system_impl.cpp:62] System initialized
I0505 20:33:48.067421   609 camera_96tof1.cpp:78] Initializing camera
I0505 20:33:48.067448   609 local_device.cpp:158] Opening device
I0505 20:33:49.010562   609 calibration.cpp:261] EEPROM calibration data size 67832 bytes
I0505 20:33:56.080806   609 camera_96tof1.cpp:95] Camera initialized
W0505 20:33:56.089836   609 local_device.cpp:375] Setting Pixel Format error, errno: 22 error: Invalid argument
W0505 20:33:56.090103   609 camera_96tof1.cpp:290] Failed to set frame type
E0505 20:33:56.090140   609 main.cpp:76] Could not set camera frame type!

the /dev/video0 device exists whether the camera is connected or not, so I think there is an issue with the csi interface, either the kernel or the device tree. Remember I am using a stock IMX8 4.19.98 kernel, not the ADDI raspberry pi kernel.

root@imx8mm-var-dart:/sys/devices/platform/32e20000.csi1_bridge/video4linux/video0# cat *
81:0
0
cat: device: Is a directory
0
mx6s-csi
cat: power: Is a directory
cat: subsystem: Is a directory
MAJOR=81
MINOR=0
DEVNAME=video0
root@imx8mm-var-dart:/sys/devices/platform/32e20000.csi1_bridge/video4linux/video0# ls -alrt
total 0
lrwxrwxrwx 1 root root    0 May  5 20:32 subsystem -> ../../../../../class/video4linux

I'm not really sure where to look next. suggestions?

[btogorean]

Hi, Can you please post the output of v4l2-ctl --all , v4l2-ctl --info and v4l2-ctl --list-formats-ext

The code is failing here So width, height or buffer type (V4L2_CAP_VIDEO_CAPTURE or V4L2_CAP_VIDEO_CAPTURE_MPLANE) are not accepted by the csi driver.

[bitslip6]

Thanks for the late night reply :)

I can't help but notice this is the first xioctl on the device and likely there is setup or communication problem unrelated to video width/height but rather any ioctl call.

Please see the following:

root@imx8mm-var-dart:~# v4l2-ctl --all
Driver Info (not using libv4l2):
    Driver name   : mx6s-csi
    Card type     : i.MX6S_CSI
    Bus info      : platform:32e20000.csi1_bridge
    Driver version: 4.14.98
    Capabilities  : 0x84200001
        Video Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps   : 0x04200001
        Video Capture
        Streaming
        Extended Pix Format
Priority: 0
Video input : 0 (Camera: ok)
Format Video Capture:
    Width/Height      : 0/0
    Pixel Format      : ''
    Field             : Any
    Bytes per Line    : 0
    Size Image        : 0
    Colorspace        : Default
    Transfer Function : Default
    YCbCr/HSV Encoding: Default
    Quantization      : Default
    Flags             : 
Crop Capability Video Capture:
    Bounds      : Left 0, Top 0, Width 0, Height 0
    Default     : Left 0, Top 0, Width 0, Height 0
    Pixel Aspect: 1/1
Crop: Left 0, Top 0, Width 0, Height 0

root@imx8mm-var-dart:~# v4l2-ctl --info
Driver Info (not using libv4l2):
    Driver name   : mx6s-csi
    Card type     : i.MX6S_CSI
    Bus info      : platform:32e20000.csi1_bridge
    Driver version: 4.14.98
    Capabilities  : 0x84200001
        Video Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps   : 0x04200001
        Video Capture
        Streaming
        Extended Pix Format

root@imx8mm-var-dart:~# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT

[btogorean]

Hi Cory,

It looks like the VIDIOC_ENUM_FMT ioctl is returning null so the frame format get/set is not working as expected. I think it all starts here and it should propagate to addi9036_enum_mbus_code but for some reason this mechanism is not working.

Unfortunately I can't point out exactly the source of the problem. I recommend you to try to debug using printks or post a question on NXP or Variscite forum maybe someone hit the same problem.

[bitslip6]

Thanks for tracking this down. I'll do some debugging and get back to you later today / tomorrow with additional information....

[bitslip6]

I have done some additional research and it looks like all ioctl calls are failing.

in v4l2-ioctl.c : __video_do_ioctl() the call is passing the pass is_known_ioctl(cmd) check and both !test_bit() and and v4l2_prio_check() tests pass. I added some error logging (here) and replaced it with this:

write_only = _IOC_DIR(cmd) == _IOC_WRITE;
    if (info->flags & INFO_FL_STD) {
        typedef int (*vidioc_op)(struct file *file, void *fh, void *p);
        const void *p = vfd->ioctl_ops;
        const vidioc_op *vidioc = p + info->u.offset;

        ret = (*vidioc)(file, fh, arg);
        pr_err("%s: fl_std ioctl and got [%ld]\n", video_device_node_name(vfd), ret);
        pr_err("%s: fl_std ioctl and argggg [%pR]\n", video_device_node_name(vfd), arg);
    } else if (info->flags & INFO_FL_FUNC) {
        ret = info->u.func(ops, file, fh, arg);
        pr_err("%s: fl_func ioctl and got [%ld]\n", video_device_node_name(vfd), ret);
        pr_err("%s: fl_func ioctl and argggg [%pR]\n", video_device_node_name(vfd), arg);
    } else if (!ops->vidioc_default) {
        pr_err("%s: vidioc_default ioctl returning error [%ld]\n", video_device_node_name(vfd), ret);
        ret = -ENOTTY;
    } else {
        ret = ops->vidioc_default(file, fh,
            vfh ? v4l2_prio_check(vfd->prio, vfh->prio) >= 0 : 0,
            cmd, arg);
        pr_err("%s: vidioc_default ioctl and got [%ld]\n", video_device_node_name(vfd), ret);
        pr_err("%s: fl_func ioctl and argggg [%pR]\n", video_device_node_name(vfd), arg);
    }

when I run the first frame application, this is what I see in the kernel log: [ 112.496236] video0: fl_func ioctl and got [-22] [ 112.499474] video0: fl_func ioctl and argggg [[??? 0x00000001-0x3c000000280 flags 0x0]]

ultimately, when the SDK tries to bring up the camera interface it blows up [here](struct mx6s_fmt *format_by_fourcc(int fourcc)): when it is called from mx6s_videoc_try_fmt_vid_cap

I have verified with some kernel logging that the driver has 4 valid formats, but the function format_byfourcc is passed 0x00. (from my extended kernel logging):_

[  112.468670] format_by_fourcc: called from mx6s_vidioc_try_fmt_vid_cap+0x7c/0x208
[  112.474773] pix format num formats: 4
[  112.477130] pix format: 0x59565955
[  112.479259] pix format: 0x56595559
[  112.481356] pix format: 0x34565559
[  112.483465] pix format: 0x31384142
[  112.485563] unknown pixelformat:'    ' 0x0

This results in the csi driver calling: "mx6s_csi_close" and eventually blowing up the kernel module:

[  112.543918] ------------[ cut here ]------------
[  112.548902] WARNING: CPU: 3 PID: 635 at /usr/src/debug/addi-mod/0.1-r0/addi9036.c:222 addi9036_s_power+0xe8/0x140 [addi9036]
[  112.558811] Modules linked in: input_leds usbhid bnep bluetooth ecdh_generic ci_hdrc_imx ci_hdrc ehci_hcd udc_core usbcore usb_common leds_gpio led_class phy_generic fec gpio_keys usbmisc_imx ads7846 mousedev addi9036(O) galcore(O) ipv6 autofs4 [last unloaded: cfg80211]
[  112.581428] CPU: 3 PID: 635 Comm: first-frame Tainted: G        W  O    4.14.98-imx8mm+ga9b5dcb9ef09 #1
[  112.589514] Hardware name: Variscite DART-MX8MM board (DT)
[  112.593694] task: ffff80007a9a8c80 task.stack: ffff000013730000
[  112.598311] PC is at addi9036_s_power+0xe8/0x140 [addi9036]
[  112.602580] LR is at addi9036_s_power+0x28/0x140 [addi9036]
[  112.606847] pc : [<ffff0000008be1b8>] lr : [<ffff0000008be0f8>] pstate: 60000145
[  112.612935] sp : ffff000013733d40
[  112.614944] x29: ffff000013733d40 x28: ffff80007a9a8c80 
[  112.618952] x27: ffff0000087b1000 x26: 0000000000000039 
[  112.622960] x25: 0000000000000124 x24: ffff80007a722038 
[  112.626968] x23: ffff80007b992000 x22: 0000000000000000 
[  112.630975] x21: ffff80007b277e98 x20: ffff80007b277c18 
[  112.634983] x19: ffff80007b277c28 x18: 0000000000008000 
[  112.638991] x17: 0000ffffb0dd57a8 x16: ffff0000081c19a8 
[  112.642998] x15: 00000000fffffff0 x14: ffff000008afc538 
[  112.647005] x13: ffff000008b9fdf0 x12: ffff000008ae8930 
[  112.651013] x11: ffff000008453398 x10: ffff000008b9fdb8 
[  112.655021] x9 : 0000000000000006 x8 : 20656c6261736964 
[  112.659029] x7 : 5f6b6c635f697363 x6 : 0000000000000200 
[  112.663036] x5 : 0000000000000000 x4 : 0000000000000000 
[  112.667043] x3 : 0000000000000000 x2 : 0000000000000000 
[  112.671050] x1 : 0000000000000000 x0 : 00000000ffffffff 
[  112.675058] Call trace:
[  112.676200] Exception stack(0xffff000013733c00 to 0xffff000013733d40)
[  112.681336] 3c00: 00000000ffffffff 0000000000000000 0000000000000000 0000000000000000
[  112.687861] 3c20: 0000000000000000 0000000000000000 0000000000000200 5f6b6c635f697363
[  112.694385] 3c40: 20656c6261736964 0000000000000006 ffff000008b9fdb8 ffff000008453398
[  112.700909] 3c60: ffff000008ae8930 ffff000008b9fdf0 ffff000008afc538 00000000fffffff0
[  112.707433] 3c80: ffff0000081c19a8 0000ffffb0dd57a8 0000000000008000 ffff80007b277c28
[  112.713957] 3ca0: ffff80007b277c18 ffff80007b277e98 0000000000000000 ffff80007b992000
[  112.720481] 3cc0: ffff80007a722038 0000000000000124 0000000000000039 ffff0000087b1000
[  112.727005] 3ce0: ffff80007a9a8c80 ffff000013733d40 ffff0000008be0f8 ffff000013733d40
[  112.733530] 3d00: ffff0000008be1b8 0000000060000145 ffff000013733d20 ffff80007a9a8c80
[  112.740055] 3d20: 0000ffffffffffff ffff80007a9a8c80 ffff000013733d40 ffff0000008be1b8
[  112.746581] [<ffff0000008be1b8>] addi9036_s_power+0xe8/0x140 [addi9036]
[  112.751895] [<ffff00000857d164>] mx6s_csi_close+0x8c/0xc8
[  112.755991] [<ffff000008552a0c>] v4l2_release+0x3c/0x90
[  112.759914] [<ffff0000081c72c0>] __fput+0x88/0x1c8
[  112.763401] [<ffff0000081c745c>] ____fput+0xc/0x18
[  112.766888] [<ffff0000080bd9dc>] task_work_run+0x9c/0xc0
[  112.770896] [<ffff000008088f0c>] do_notify_resume+0xfc/0x108
[  112.775249] Exception stack(0xffff000013733ec0 to 0xffff000013734000)
[  112.780384] 3ec0: 0000000000000000 0000ffffb0df0000 0000000000000000 0000000000000000
[  112.786908] 3ee0: 0000000000000000 0000000000000004 0000000000000001 0000000000003d6e
[  112.793432] 3f00: 0000000000000039 003b9aca00000000 0101010101010101 0000000000000000
[  112.799956] 3f20: 0000000000000018 00000003e8000000 0000ffffb0dc7c38 0000ffffb0dc5dd8
[  112.806480] 3f40: 0000ffffb112e2e0 0000ffffb0dd57a8 0000000000000061 0000000000000005
[  112.813005] 3f60: 0000000023acf690 00000000004041c0 0000000000000000 0000000000000000
[  112.819529] 3f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  112.826053] 3fa0: 0000000000000000 0000ffffd86d41e0 0000ffffb0f797a4 0000ffffd86d41e0
[  112.832577] 3fc0: 0000ffffb0dd57d0 0000000060000000 0000000000000005 0000000000000039
[  112.839101] 3fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  112.845625] [<ffff000008083164>] work_pending+0x8/0x10
[  112.849458] ---[ end trace 79e918363cf4f7fa ]---

I believe this may be a physical signal problem, either the device tree is mapping the wrong pins, or we have the wrong pins pulled out to the AFE.

It is important to note that the /dev/video0 device is "up" and available if the AFE is connected or is not connected.

question 1: should the ioctl call actually signal the AFE or does this just return information from the driver? (is the failed ioctl a result of AFE communication failure or just kernel driver failure)

question 2: any additional ideas on debugging this issue?

Thanks so much for your help!

[acozma]

Here there is an older SDK port to the Thor96 board which has an IMX.8 processor. Maybe this will be of some help.

[bitslip6]

Thanks for the reply!

No, we have the SDK up and running. The issue is in the kernel module. Is there anyway we can schedule a skype call or something similar?

[bitslip6]

Thanks for the patch. That has in deed corrected the pixel format issue.

We are now blowing up at local_device.cpp line 471 (I added 2 lines at 358,359 for the pixel format from the patch you linked so your line numbers may be a bit off). It's from the second xioctl for VIDIOC_S_EXT_CTRLS

W0529 00:39:45.279739 1322 local_device.cpp:471] Programming AFE error errno: 25 error: Inappropriate ioctl for device

I added additional logging to the kernel and have isolated it to line 2699 in v4l2-ioctl.c here: https://github.com/varigit/linux-imx/blob/427eb8371fd5e9bceec18ca8838ac6eb1e51ad3a/drivers/media/v4l2-core/v4l2-ioctl.c#L2699

It looks like the video device does not know about the VIDIOC_S_EXT_CTRLS call.

I think we are close. just missing one or 2 more patches...

[btogorean]

Please take care that this commit add also a customm IOCTL for AFE programming. I think you miss that one

[bitslip6]

The ADDI9036 drive is compiled as a loadable kernel module. That is working correctly. As you can see from the initial ticket message. I have attached our kernel build patch (which includes device tree changes for addi9036, spi dev (tft display), additional debugging log messages in the imx6csi driver and the addi9036 driver) if that helps.

I know the i2c bus is working. Perhaps there is a problem with the csi clock or data lines?

kernel_patch.zip

[bitslip6]

Can I get an update on this? It's been quite a while....

[dNechita]

Hi,

Our linux/device tree guru is on vacation and will return on June 9th.

[btogorean]

Hi Cory,

First I would like to apologize for my very late response. I was in vacation as Dan mentioned before.

I think the problem is that /dev/video0 is not the device we actually require. As you can see here Thor96 device enumerator there is created a subdevice for ADI TOF camera. So the VIDIOC_S_EXT_CTRLS are associated and handled by the child subdevice and not the root video0 device entry.

Please check what new entries are created in /dev when you probe the camera module and check their capabilities using v4l2-ctl -l -d /dev/v4l-subdevX command. When you find the correct one it should have the chip_config and reg_read Image Processing Controls listed.

After that modify your device_enumerator.cpp in SDK accordingly.

[bitslip6]

here is the addi dmesg: root@imx8mm-var-dart:/dev/v4l/by-path# dmesg | grep addi [ 3.961477] addi9036 3-0064: addi9036_probe: i2c addr = 0x64 [ 4.005335] addi9036 3-0064: addi9036 detected at address 0x64 [ 4.095141] addi9036 3-0064: set_fmt: 0 640x960

and here is the subdevice file listing: root@imx8mm-var-dart:/dev/v4l/by-path# ls -lart total 0 lrwxrwxrwx 1 root root 12 Apr 28 17:37 platform-32e20000.csi1_bridge-video-index0 -> ../../video0 drwxr-xr-x 3 root root 60 Apr 28 17:37 .. drwxr-xr-x 2 root root 60 Apr 28 17:37 .

doesn't look like anything is registering :(

here is the i2c status: root@imx8mm-var-dart:/dev/v4l/by-path# i2cdetect -y 3 0 1 2 3 4 5 6 7 8 9 a b c d e f 00: -- -- -- -- -- -- -- -- -- -- -- -- -- 10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 50: -- -- -- -- -- -- UU UU -- -- -- -- -- -- -- -- 60: -- -- -- -- UU -- -- -- -- -- -- -- -- -- -- -- 70: -- -- -- -- -- -- -- --

all i2c interfaces are working, but no video. I'm thinking perhaps this is a hardware issue? We have clock0P/N data0P/N and data1P/N lines connected from the our relevant device tree diff from arch/arm64/boot/dts/freescale/fsl-imx8mm-var-dart.dts:

23 @@ -821,11 +823,44 @@ 24 pinctrl-names = "default"; 25 pinctrl-0 = <&pinctrl_i2c4>; 26 status = "okay"; 27 + 28 + eeprom_i2c4: eeprom@56 { 29 + compatible = "atmel,24c1024"; 30 + reg = <0x56>; 31 + pagesize = <32>; 32 + }; 33 + 34 + camera_front@64 { 35 + compatible = "adi,addi9036"; 36 + reg = <0x64>; 37 + status = "okay"; 38 + 39 + pinctrl-names = "default"; 40 + pinctrl-0 = <&pinctrl_csi1>; 41 + reset-gpios = <&gpio2 10 1>; 42 + 43 + csi_id = <0>; 44 + clock-names = "csi_mclk"; 45 + 46 + rst-gpios = <&gpio2 10 GPIO_ACTIVE_LOW>; 47 + port { 48 + addi9036_ep: endpoint { 49 + remote-endpoint = <&csi1_ep>; 50 + clock-lanes = <0>; 51 + data-lanes = <1 2>; 52 + }; 53 + }; 54 + }; 55 + 56 };

The IMX8 and AFE data line connections are based on schematics here: https://pasteboard.co/JcjmFR2.png https://pasteboard.co/JcjncD2.png

currently i2c clk/data, clock P/N, data0 P/N, data1 P/N and ground are connected. There is no reset gpio connection. Do you know if this is required? I don't see any reset lines on the AFE schematics, I assume any reset can be done using the i2c interface?

Thanks for all of your help on this. It's very appreciated!

Best regards, Cory

[bitslip6]

As a side note, hooking up my scope to the clock and data lines shows no activity when running the first-frame application.

[btogorean]

Hi Cory,

Subdevices are not listed in /dev/v4l/. You can search for them using media-ctl -p or search in /dev for v4l-subdev

I2Cdetect output looks OK and also devicetree snippet you posted looks ok.

Please take care that if SDK is not passing the "Programming AFE" step no activity on clock or data lines is visible so I say that we can't conclude right now that is a HW issue.

[bitslip6]

HI Bogdan! Thanks for the reply. Was just about to go to go to sleep...

root@imx8mm-var-dart:~/aditof_sdk/build/examples/first-frame# v4l2-ctl --list-devices i.MX6S_CSI (platform:32e20000.csi1_bridge): /dev/video0

root@imx8mm-var-dart:~/aditof_sdk/build/examples/first-frame# v4l2-ctl --device=/dev/video0 --all Driver Info (not using libv4l2): Driver name : mx6s-csi Card type : i.MX6S_CSI Bus info : platform:32e20000.csi1_bridge Driver version: 4.14.98 Capabilities : 0x84200001 Video Capture Streaming Extended Pix Format Device Capabilities Device Caps : 0x04200001 Video Capture Streaming Extended Pix Format Priority: 0

Message from syslogd@imx8mm-var-dart at Wed Jun 10 06:26:03 2020 ... imx8mm-var-dart kernel: Internal error: Oops: 86000006 [#1] PREEMPT SMP

Kernel Panic :(

Have time for a call? email: cory@bitslip6.com

[bitslip6]

Thanks for you time today! I have tracked this issue down to v4l2_device_register_subdev_nodes not ever being called. this is the call that registers the subdev device entries in /dev/

this post: https://community.nxp.com/thread/500322 suggests the imx8 doesn't have a hook for this, so they call it in mipi_csi2_s_stream, however I don't see the call to v4l2_device_register_subdev_nodes in any of the 4.14.98 kernel code we have from variscite.

all of the other calls to register_subdev_nodes in our kernel are in endpoint drivers. This method requires a v4l2_device entry which the addi9036 driver does not have.

I don't see how this ever worked on any other Linux unless the call to v4l2_device_register_subdev_nodes is handled by the csi driver on the other platforms.

I'll continue to trace this down today...

[bitslip6]

so I got the subdevice up and working by adding the line: ret = v4l2_device_register_subdev_nodes(addi9036->sd.v4l2_dev);

after the call to ret = v4l2_async_register_subdev(&addi9036->sd); in addi9036_probe

everything goes along just fine until the call to csi start streaming that calls csi enable and then addi9036_s_ctrl() and set_chip_config

set chip config runs 2 times then on the 3rd time I get the error could not write to register which then repeats about 650 times and then calls csi stop streaming

I'll continue investigating today. If you have any suggestions please send them my way.

[btogorean]

Hi Cory,

We succeed starting the aditof-demo on our NXP I.MX8 Mini-evk board. The camera driver used is the one from our linux repo Please see the diff that includes all the required changes applied over imx_4.14.78_1.0.0_ga kernel tree.

imx8mm.txt

We used Arrow electronics fork of ADI Tof SDK that can be found here On SDK the i2c eeprom path should be updated accordingly to your configuration in target_definitions.h and on device_enumerator.c we have: devData.driverPath = /dev/video0;/dev/v4l-subdev0";

I hope with these changes you'll be able to start your setup too.

[bitslip6]

First of all thank you for the patch and the call last week! We have the subdevice working correctly however set_chip_config runs 2x then tries to program the AFE and the call to is_writable_register fails attempting to write 0x0 to address 0x0. I have verified clock and data on the CSI interface with my scope and they seem to be functioning correctly as well as the i2c interface so I don't think it is hardware related right now.

I'll walk through the SDK code and see which call exactly is failing this week and update you guys. I have also merged in your patch with our work so we are running the same code but I am still seeing the error on the third call to set_chip_config.

[btogorean]

Good to hear that Cory,

The is_writable_register will fail because here in regmap we define the write allowed register space. As you can see 0x0 is not in the allowed range. But when AFE is programmed from SDK one programming transaction is interpreted as write to address 0x0 and regmap will complain with an error. Is a known issue and we'll try to fix it. But besides this the camera should work fine.

Do you have now image on aditof_demo or not?

[bitslip6]

Sorry for the late reply. My camera here died shortly after our call so i have been waiting on new hardware. static electricity maybe?

Thanks for the patch. Your version is a bit cleaner that the patch I proposed last week. it is currently applied to our kernel and all kernel debug messages have been removed.

our custom boards came in so I am attempting to get everything running on that system. the schematics are the same as the dev kit, so we think it should be okay. I may move back to the stock dev kit now that I have new stock hardware to replace the dead kit.

I'm working on getting frame data out, but I'm getting a select timeout here: W0625 20:29:59.104471 515 local_device.cpp:879] select timeout W0625 20:29:59.104719 515 camera_96tof1.cpp:316] Failed to get frame from device

When looking at the CSI clock line i see a pulse when the application attempts to request frame data but just one pulse, clock goes high for a moment (+1.5v) and then that seems to be it. My scope is only 100Mhz, so it's a little difficult to tell exactly. our reset line is currently floating. any suggestions for where to look?

As always, thanks for your help, very much apprecuited!

Cory

[acozma]

Can you please try with the stock dev kit to see if the issue persists? One of the causes might be that the device is still not programmed correctly. On the dev kit it will be easy to see if that's the case by looking at the LEDs on top right of the camera board which should all light up after programming.

[bitslip6]

Hi andri, thanks for the quick reply. So I have a little more data. I am using the stock kit, I have verified the eeprom value is correct with hexdump. The programming goes correctly I can see the i2c bus on the csi interface writing a lot of data after reading the eeprom. then I hear the stock kit start switching (presumably the leds are working), all leds are lit up (DS1, DS2, DS3)

I have looked at the clock N and clock P lines on the csi interface on my scope. Unfortunately I only have a 100Mhz scope at home. My colleague with a 4Ghz scope sees clock pulses every 5ns from 300mV to 100mV.

On my home scope I see about 200mV ripple with a mean voltage of 200mV after the AFE programming completes. so I cannot see the actual pulses, but everything indicates that the clock is working correctly.

Any suggestions for where to add some additional kernel debug messages? perhaps there is some additional kernel call that must be made to start accepting data on the csi interface?

Thanks much for all of your help!

[acozma]

Hi @bitslip6

From what we saw, on the imx8 there doesn't seem to be necessary to do additional video pipeline configuration, but still investigating for this particular case. Can you please post the output of dmesg after starting the video stream? Maybe that will give an indication of why no data is coming out. Since the clock is active it means that the hardware is properly configured.

[bitslip6]

Hi! So I have verified data on all clock and data lines. The issue is definitely software. I'm going to reset to master on kernel , kernel module and sdk and just add the changes for subdev_notifier_complete. I'll post in an hour or so when I have results...

[bitslip6]

I have removed all changes (mostly debug messages) from addi9036.c and I am using the version you posted.

I am still getting the timeout. I have verified all 6 clock and data lines and each looks fine. all i2c buses work fine. v4l2 sub-device is created fine. I am loading the addi9036 driver as a kernel module. I have attached the bitbake recipe for the kernel module as well as our kernel source and device tree. any help would be most appreciated! addi-module.tar.gz kernel-recipe.tar.gz

dmesg: Jun 30 06:00:30 imx8mm-var-dart kernel: IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready Jun 30 06:00:36 imx8mm-var-dart kernel: IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready Jun 30 06:09:35 imx8mm-var-dart kernel: addi9036 3-0064: addi9036 detected at address 0x64 Jun 30 06:09:35 imx8mm-var-dart kernel: mxc_mipi-csi 32e30000.mipi_csi: Registered sensor subdevice: addi9036 3-0064 Jun 30 06:10:11 imx8mm-var-dart kernel: addi9036 3-0064: could not write to register 0 Jun 30 06:10:11 imx8mm-var-dart last message repeated 368 times Jun 30 06:10:11 imx8mm-var-dart kernel: <44>systemd-journald[187]: /dev/kmsg buffer overrun, some messages lost. Jun 30 06:10:11 imx8mm-var-dart kernel: addi9036 3-0064: could not write to register 0 Jun 30 06:10:11 imx8mm-var-dart last message repeated 27 times Jun 30 06:10:11 imx8mm-var-dart kernel: <44>systemd-journald[187]: /dev/kmsg buffer overrun, some messages lost. Jun 30 06:10:11 imx8mm-var-dart kernel: addi9036 3-0064: could not write to register 0 Jun 30 06:10:11 imx8mm-var-dart last message repeated 26 times

[acozma]

Can you please post the output of dmesg after starting the video stream? Need to see what's going on after the streamon command is issued.

[bitslip6]

when this happens, the stream on command is not being called. I have identified this is the issue. I'll post update later today.

[bitslip6]

so I'm getting data from the sensor now. it looks like random contents of memory however.

here is an image i produced from the captured data on our tft display. the output is the result of:

// pseudo code: int value = data[((y*640)+x] >>4 ; // the value of the current pixel location "downscaled" to a 0-255 range int 16bit_uint_color = hsv_ro_rgb(PURPLE, 200, value); // PURPLE = hue, 200 = saturation, value = depth in 0-255 range plot(x, y, color);

any ideas on what could be causing this? the driver memset(0) the memory before writing the sensor data correct?

I have verified with my 100Mhz scope here data on all clock and data lines. our team in Germany will do some additional testing with their 4Ghz scope tomorrow. any suggestions would be appreciated!

[acozma]

This looks very random. Is there any change when moving an object in front of the camera?

Could you save a few raw frames, before deinterleaving, in this function https://github.com/analogdevicesinc/aditof_sdk/blob/master/sdk/src/connections/mipi/local_device.cpp#L500 and post the file?

[bitslip6]

Hi Andrei thanks for the reply. I was on vacation end of last week. a few updates.

This is using the stock adi dev kit. All three lights are green and dmesg output show no errors at all (just a lot of kernel trace logging that I added). I see data on the CSI clock and data lines on my scope and it looks good. The image displayed above is simply uninitialized memory. I get the exact same frame output on each call to getFrame(). If I restart the IMX8 or the ADDI kit then I get a different image, but the image is always the same for every call to getFrame().

Could this be a problem with the image de-interleaving in the kernel? We have a stock kernel except for some trace level logging messages and this change in mx6s_capture on the mx6s_csi_enable function:

 if (pix->field == V4L2_FIELD_INTERLACED)
     csi_tvdec_enable(csi_dev, true);

• else {
• printk("video not interlaced\n");
• csi_tvdec_enable(csi_dev, false);
• }

[bitslip6]

I have confirmed that this line is not hit: https://coral.googlesource.com/linux-imx/+/refs/tags/4-2/drivers/media/platform/mxc/capture/mx6s_capture.c#758

with the stock driver the call to csi_tvdec_enable is never made and I receive a select timeout. With the modified driver (using above patch) I receive the "matrix" image as seen above. Seems to be a problem with the image formatting on imx8 csi kernel driver.

Any input is appreciated. Thanks!

[bitslip6]

here is the full diff of the imx8 build for reference.

I have tried both the arrow fork and main sdk and all tests have been with the stock adi dev kit hardware.

imx8diff.txt

[bitslip6]

can you send me a complete gzip of the kernel source that you got working on the imx8? I believe there are some differences in the imx csi driver I would like to merge into our kernel. I think this is the key.

Thanks so much for your help on this!