search

VC-MIPI-modules / vc_mipi_nvidia

Vision Components MIPI CSI-2 driver for NVIDIA Jetson Nano, Xavier NX, AGX Xavier, TX2 and Orin Nano, Orin NX
84 stars 34 forks source link

Patch failled for L4T 35.3.1 #88

Open noob502 opened 6 months ago

noob502 commented 6 months ago

I am creating an image for Jetson Orin Nano using Yocto+Openembedded-core build tools. I am following this link ( https://github.com/VC-MIPI-modules/vc_mipi_nvidia#integrate-the-driver-in-your-own-bsp ) to add vc_mipi into the image.

When I am trying to apply the patch, it asks the 'File to patch', so I share the related file path, but it fails a lot of times and saves them into a reject file. Later, I opened those reject files and manually modified the related files (adding lines or removing lines according to the reject file).

Despite my efforts to manually input the suggested lines from the reject file, the patch application continues to show a high number of Failed warnings. It consistently displays the same issue, even after I've made the suggested modifications.

For example, I am sharing a small implementation: 

noob@noober:~/surface-linux-os-l3531/build/workspace/sources/linux-tegra$ patch -p1 < 0001-Added-RAW8-grey-RAW10-y10-RAW12-y12-and-RAW14-y14-fo.patch 
(Stripping trailing CRs from patch; use --binary to disable.)
patching file kernel/kernel-4.9/Documentation/media/uapi/v4l/pixfmt-y14.rst
(Stripping trailing CRs from patch; use --binary to disable.)
can't find file to patch at input line 95
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/kernel/kernel-4.9/Documentation/media/uapi/v4l/yuv-formats.rst b/kernel/kernel-4.9/Documentation/media/uapi/v4l/yuv-formats.rst
|index 3334ea445..b2e7e4a57 100644
|--- a/kernel/kernel-4.9/Documentation/media/uapi/v4l/yuv-formats.rst
|+++ b/kernel/kernel-4.9/Documentation/media/uapi/v4l/yuv-formats.rst
--------------------------
File to patch: /home/noob/surface-linux-os-l3531/build/workspace/sources/linux-tegra/kernel/kernel-4.9/Documentation/media/uapi/v4l/pixfmt-y14.rst
patching file /home/u26000047/surface-linux-os-l3531/build/workspace/sources/linux-tegra/kernel/kernel-4.9/Documentation/media/uapi/v4l/pixfmt-y14.rst
Hunk #1 FAILED at 28.
1 out of 1 hunk FAILED -- saving rejects to file /home/noob/surface-linux-os-l3531/build/workspace/sources/linux-tegra/kernel/kernel-4.9/Documentation/media/uapi/v4l/pixfmt-y14.rst.rej
(Stripping trailing CRs from patch; use --binary to disable.)
can't find file to patch at input line 107
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c b/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c
|index 0c559476b..1173ef56e 100644
|--- a/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c
|+++ b/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c
--------------------------
File to patch:

Am I applying the patch the right way? If so, then why failed every time?

bazo80 commented 6 months ago

Hello @noob502 ,

you are trying to apply the patches from kernel-4.9 into a L4T35.3.1 (kernel-5.10)? You should use the patches from kernel_Xavier_35.3.1+ directory, because the source changes between 4.9 and 5.10 are too serious.

Regards

noob502 commented 6 months ago

Hi @bazo80 ,

I am using kernel_common_32.3.1 and kernel_Xavier_35.3.1 together. Four patches from kernel_common_32.3.1 can be applied without error, but patches from kernel_Xavier_35.3.1 are showing errors.

Best Regards

bazo80 commented 6 months ago

Hello @noob502 ,

I'm just wondering, because your console output is stating something else.

patching file kernel/kernel-4.9/Documentation/media/uapi/v4l/pixfmt-y14.rst
...
|diff --git a/kernel/kernel-4.9/Documentation/media/uapi/v4l/yuv-formats.rst b/kernel/kernel-4.9/Documentation/media/uapi/v4l/yuv-formats.rst
|index 3334ea445..b2e7e4a57 100644
...
|diff --git a/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c b/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c
|index 0c559476b..1173ef56e 100644

This is from the file 0001-Added-RAW8-grey-RAW10-y10-RAW12-y12-and-RAW14-y14-fo.patch, located in the directory kernel_Xavier_32.3.1+ (the indices 3334ea445 and 0c559476b are existing only in .../kernel_Xavier_32.3.1+/0001-Added-RAW8-grey-RAW10-y10-RAW12-y12-and-RAW14-y14-fo.patch), but you should use the files from the directory kernel_Xavier_35.3.1+

Regards

noob502 commented 6 months ago

Hi @bazo80, That means I should use patches only from kernel_Xavier_35.3.1+ ?

bazo80 commented 6 months ago

Hello @noob502,

you stated, that the patches from kernel_common have been applied correctly. That is ok, because the code base from the four patches hasn't changed during the L4T versions. But there is also code, which is specific to the different L4T versions. In your case 35.3.1. That means, you need the additional patches from the folder kernel_Xavier_35.3.1+ not kernel_Xavier_32.3.1+

The patch folders kernel_*_xx.y.z+ are containing patch files, which might have the same name, but the content of the files are different.

In your example, the file 0001-Added-RAW8-grey-RAW10-y10-RAW12-y12-and-RAW14-y14-fo.patch has not been applied, because this patch file was from kernel_Xavier_32.3.1+ folder. You should have used the file kernelXavier35.3.1+/0001-Added-RAW8-grey-RAW10-y10-RAW12-y12-and-RAW14-y14-fo.patch.

Anyway, the following patches must be applied for your case:

kernel_common_32.3.1+/0001-Added-cropping-position-left-top-to-sensor-image-pro.patch kernel_common_32.3.1+/0001-Added-.gitignore.patch kernel_common_32.3.1+/0001-Added-implementation-to-set-image-position-and-size-.patch kernel_common_32.3.1+/0002-Reduced-image-size-limitation-from-width-32-to-4-and.patch

kernelXavier35.3.1+/0001-Added-controls-trigger_mode-io_mode-black_level-and-.patch kernelXavier35.3.1+/0001-Added-RAW8-grey-RAW10-y10-RAW12-y12-and-RAW14-y14-fo.patch kernelXavier35.3.1+/0001-Added-VC-MIPI-Driver-sources-to-Makefile.patch kernelXavier35.3.1+/0001-Added-VC-MIPI-driver-to-Kconfig.patch kernelXavier35.3.1+/0001-Disable-VB2_BUF_STATE_REQUEUEING-in-vi5_fops.c.patch kernelXavier35.3.1+/0001-Increased-tegra-channel-timeout.patch kernelXavier35.3.1+/0001-Modified-tegra234-p3509-a02.dtsi-to-integrate-VC-MIP.patch kernelXavier35.3.1+/0001-Modified-tegra234-p3768-0000-a0.dts-to-integrate-VC-.patch kernelXavier35.3.1+/0003-Changed-Interrupt-Mask-for-csi4-to-emit-CRC-and-mul.patch kernelXavier35.3.1+/0001-Stability-patch.-work_struct-refactored-to-kthread.patch

The following files can be left out in your case, because they are not Orin related: kernelXavier35.3.1+/0001-Modified-tegra194-p2888-0001-p2822-0000.dts-to-integ.patch kernelXavier35.3.1+/0001-Modified-tegra194-p3509-0000-a00.dtsi-to-integrate-V.patch

regards

noob502 commented 6 months ago

I applied the patches, now I can not see /dev/video0 or /dev/video1 but without applying the patches I can see my camera (/dev/video0) into the board. Am I missing something ?

Note: I am building minimal / full_line_cmd / sato image and then added the vc_mipi into it.

Here are the packages I added gradually for further development

IMAGE_INSTALL:append= " cuda-driver tegra-libraries-cuda gcc python3 \ v4l-utils util-linux gstreamer1.0-plugins-nvvideo4linux2\ apt dpkg sudo coreutils tegra-mmapi tegra-mmapi-dev\ i2c-tools \ "

bazo80 commented 6 months ago

Hello @noob502 ,

what is your dmesg output? Do you see the output of the vc_mipi driver like such a table?

 +--- VC MIPI Camera -----------------------------------+
 | MANUF. | Vision Components               MID: 0x0427 |
 | MODULE | ID:  0x0567                     REV:   0002 |
 | SENSOR | SONY IMX567                                 |
 +--------+---------------------------------------------+
 +--- Sensor Registers ------+--------+--------+--------+
 |                           | low    | mid    | high   |
 +---------------------------+--------+--------+--------+
...

Has the device tree file being applied correctly? dmesg | grep "DTS File Name" This command should give you the applied device-tree. Does this file have 2 vc_mipi entries? Has the file tegra234-camera-vc-mipi-cam.dtsi been configured with 2 sensors?

best regards

noob502 commented 5 months ago

@bazo80 No, I do not see such table. But here is some terminal command output

root@jetson-orin-nano-devkit:~# dmesg | grep "vc_mipi"[ 1608.714936] [ 5.238303] vc_mipi 9-001a: vc_probe(): Probing UNIVERSAL VC MIPI Driver (v0.17.0) [ 7.647688] vc_mipi 9-001a: vc_mod_setup(): Unable to get module I2C client for address 0x10 [ 7.647934] vc_mipi 9-001a: vc_probe(): Error in vc_core_init! [ 7.650331] vc_mipi 10-001a: vc_probe(): Probing UNIVERSAL VC MIPI Driver (v0.17.0) [ 10.047685] vc_mipi 10-001a: vc_mod_setup(): Unable to get module I2C client for address 0x10 [ 10.047931] vc_mipi 10-001a: vc_probe(): Error in vc_core_init! root@jetson-orin-nano-devkit:~# dmesg | grep "vc_mipi" [ 5.238303] vc_mipi 9-001a: vc_probe(): Probing UNIVERSAL VC MIPI Driver (v0.17.0) [ 7.647688] vc_mipi 9-001a: vc_mod_setup(): Unable to get module I2C client for address 0x10 [ 7.647934] vc_mipi 9-001a: vc_probe(): Error in vc_core_init! [ 7.650331] vc_mipi 10-001a: vc_probe(): Probing UNIVERSAL VC MIPI Driver (v0.17.0) [ 10.047685] vc_mipi 10-001a: vc_mod_setup(): Unable to get module I2C client for address 0x10 root@jetson-orin-nano-devkit:~# dmesg | grep "VC MIPI" [ 5.238303] vc_mipi 9-001a: vc_probe(): Probing UNIVERSAL VC MIPI Driver (v0.17.0) root@jetson-orin-nano-devkit:~# dmesg | grep "DTS File Name".17.0)
[ 0.004777] DTS File Name: tegra234-p3767-0003-p3768-0000-a0.dts

root@jetson-orin-nano-devkit:~# ls /dev/v vcs vcs2 vcs4 vcs6 vcsa1 vcsa3 vcsa5 vcsu vcsu2 vcsu4 vcsu6 vcs1 vcs3 vcs5 vcsa vcsa2 vcsa4 vcsa6 vcsu1 vcsu3 vcsu5 vhci

Do I have add tegra234-camera-vc-mipi-cam.dtsi device tree into the main device tree tegra234-p3767-0003-p3768-0000-a0.dts ? Like #include "tegra234-camera-vc-mipi-cam.dtsi" ?

As I understood, by default tegra234-camera-vc-mipi-cam.dtsi is already configured for two cameras, See this

// Driver Configuration for NVIDIA Jetson Orin Nano on NVIDIA Jetson Orin Nano Developer Kit // ------------------------------------------------------------------------------------------------

define VC_MIPI_CAM_1 1 // 1: Enabled, 0: Disabled (serial_b => CSI Port 1 => VI Stream 0)

define VC_MIPI_CAM_0 1 // 1: Enabled, 0: Disabled (serial_c => CSI Port 2 => VI Stream 2)

//

As v4l2-ctl can not find the camera, I have used media-ctl root@jetson-orin-nano-devkit:~# media-ctl -p Media controller API version 5.10.120

Media device information

driver tegra-camrtc-ca model NVIDIA Tegra Video Input Device serial
bus info
hw revision 0x3 driver version 5.10.120

Device topology

bazo80 commented 5 months ago

Hello @noob502 ,

The file Linux_for_Tegra/source/public/hardware/nvidia/platform/t23x/p3768/kernel-dts/tegra234-p3767-0003-p3768-0000-a0.dts is including the file tegra234-p3767-0000-p3768-0000-a0.dts The file Linux_for_Tegra/source/public/hardware/nvidia/platform/t23x/p3768/kernel-dts/tegra234-p3767-0000-p3768-0000-a0.dts is including the file cvb/tegra234-p3768-0000-a0.dtsi And the file Linux_for_Tegra/source/public/hardware/nvidia/platform/t23x/p3768/kernel-dts/cvb/tegra234-p3768-0000-a0.dtsi should include the file tegra234-camera-vc-mipi-cam.dtsi.

If the system is completely unpatched, the file cvb/tegra234-p3768-0000-a0.dtsi is configured to use the standard drivers

include "tegra234-p3768-camera-rbpcv3-imx477.dtsi"

include "tegra234-p3768-camera-rbpcv2-imx219.dtsi"

When the patches from kernel_Xavier_35.3.1+ have been applied correctly, the tegra234-camera-vc-mipi-cam.dtsi will replace the two standard driver includes (IMX219/IMX477) You can also have a look at the patch file 0001-Modified-tegra234-p3768-0000-a0.dts-to-integrate-VC-.patch in the folder kernel_Xavier_35.3.1+. But as far as I can see, both vc_mipi driver instances are called:

root@jetson-orin-nano-devkit:# dmesg | grep "vc_mipi"
[ 5.238303] vc_mipi 9-001a: vc_probe(): Probing UNIVERSAL VC MIPI Driver (v0.17.0)
[ 7.647688] vc_mipi 9-001a: vc_mod_setup(): Unable to get module I2C client for address 0x10
[ 7.647934] vc_mipi 9-001a: vc_probe(): Error in vc_core_init!
[ 7.650331] vc_mipi 10-001a: vc_probe(): Probing UNIVERSAL VC MIPI Driver (v0.17.0)
[ 10.047685] vc_mipi 10-001a: vc_mod_setup(): Unable to get module I2C client for address 0x10

That means, the tegra234-camera-vc-mipi-cam.dtsi is being included. That's good. But something else is going wrong.

Have you setup your system with our build pipe line and got it running once? If so, you could compare the resulting device trees. In the directory /boot/dtb/ of your running target, you can fetch the kernel_*.dtb file and copy it to your home directory. After that, you can convert this dtb into a readable form:

dtc -I dtb -O dts kernel_<DEVICE_TREE_NAME>.dtb -o kernel_<DEVICE_TREE_NAME>.dts

The same conversion can be done with your resulting dtb file of your yocto generated system. After that, you can compare both dts files. They should not differ that much. The interesting regions are the entries around the vc_mipi tree and the i2c.

best regards