search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.6k stars 4.83k forks source link

intel realsense patch for kernel 5.10 #11745

Open WallE-Bosch opened 1 year ago

WallE-Bosch commented 1 year ago
Required Info
Camera Model { D435 }
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version {Linux Ubuntu 20 and Ubuntu 22)
Kernel Version (Linux Only) (5.10)
Platform PC.
SDK Version { legacy / 2.<?>.<?> }
Language {C++ }
Segment {Robot }

Issue Description

Hello do i need the kernel patches for that kernel version 5.10 ? Or what are the right patch files for that ?

I tried to use the following patch files (v2.50.0) but it didn't work

It was not able to find the cameras

MartyG-RealSense commented 1 year ago

Hi @WallE-Bosch The correct patch script file for kernel versions older than 5.13 is patch-realsense-ubuntu-lts.sh. However, kernel 5.10 is not supported by patch-realsense-ubuntu-lts.sh. 5.8 and 5.11 are supported.

If your project is unable to upgrade the kernel version to 5.11 then you can bypass the kernel and avoid the need to kernel-patch by building the RealSense SDK from source code with CMake using the RSUSB Backend installation method described at https://github.com/IntelRealSense/librealsense/issues/9931#issuecomment-964289692

WallE-Bosch commented 1 year ago

@MartyG-RealSense Thanks for the fast answer so for 5.15 no patch is necessary? is it for 5.4 necessary?

MartyG-RealSense commented 1 year ago

If the SDK is installed from packages (which have the patch bundled in them) or built from source code with the RSUSB Backend installation method then no kernel patching is necessary.

If the SDK is built from source code but RSUSB is not used then a kernel patch should be applied if you have a kernel version that the SDK's patch scripts support. Both 5.4 and 5.15 are supported.

If a patch is not applied to a non-RSUSB source code build then there may be errors, and support for hardware metadata will not be included.

MartyG-RealSense commented 1 year ago

Hi @WallE-Bosch Do you require further assistance with this case, please? Thanks!

WallE-Bosch commented 1 year ago

So we tried out an it looks not bad

I wondered that the solution is not recommended Is there a reason for that ?

Notes: (1) RSUSB can be supported (not recommended) https://dev.intelrealsense.com/docs/build-configuration#back-end-recommended-configuration

MartyG-RealSense commented 1 year ago

RSUSB is fine for some applications but has some drawbacks that makes a kernel-patched build a better choice instead of RSUSB in certain situations. For example, RSUSB is best suited to single-camera applications rather than multicam ones.

For more information about the advantages and disadvantages of kernel-patching versus RSUSB, you can visit https://github.com/IntelRealSense/librealsense/issues/5212#issuecomment-552184604 and scroll down through the linked-to comment to the section headed What are the advantages and disadvantages of using libuvc vs patched kernel modules?

MartyG-RealSense commented 1 year ago

Hi @WallE-Bosch Do you require further assistance with this case, please? Thanks!

MartyG-RealSense commented 1 year ago

Case closed due to no further comments received.

WallE-Bosch commented 1 year ago

Thank you for support

WallE-Bosch commented 1 year ago

@MartyG-RealSense will there be a patch availible for kernel 6.1. ?

MartyG-RealSense commented 1 year ago

I will ask my Intel RealSense colleagues.

MartyG-RealSense commented 1 year ago

Hi @WallE-Bosch After discussion with my Intel RealSense colleagues, an internal Intel feature request for adding support for kernel 6.1 has been created. I have re-opened this issue and it should remain open whilst the feature request is active, but there is no further action that you need to take. Thanks!

dmipx commented 1 year ago

Hi @WallE-Bosch . We do plan to support kernel 6.1, for our patches, it is not different that 5.19. Ubuntu still not released final 6.1 kernel, when it will, we will update.

schulz-bosch commented 1 year ago

Quick follow-up question. I am supporting @WallE-Bosch with kernel topics, and we are currently experimenting with kernel version 5.15. Is it correct, that we need exactly the following patch files?

realsense-camera-formats-jammy-master.patch realsense-metadata-jammy-master.patch realsense-hid-jammy-master.patch realsense-powerlinefrequency-control-fix.patch uvcvideo_increase_UVC_URBS.patch

I'm building the jammy-kernel from scratch, using the sources from https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy on branch 'master-next'. I noticed that depending on the branch, the realsense-hid-jammy-master.patch already seems to be applied, so I'm applying all patches with patch -p1 < *.patch -N || true

Is that correct or is there anything missing? I prefer to not use the patch-realsense-ubuntu-lts-hwe.sh script, because I'm building the kernel in an enclosed build environment that differs from the host system, and because I don't want to check out the sources twice.

I also noticed that modinfo uvcvideo | grep version does not include the "realsense" string (found that here: https://github.com/IntelRealSense/librealsense/blob/development/doc/distribution_linux.md), but I suppose that's because I did not install the packages librealsense2-dkms and librealsense2-utils.

Thanks for helping us out!

MartyG-RealSense commented 1 year ago

Hi @schulz-bosch If you do not want to use a kernel patch script then you can simply bypass the kernel and not need to patch it by building the librealsense SDK from source code with CMake and including the flag -DFORCE_RSUSB_BACKEND=TRUE in the CMake build instruction. The SDK will then not be dependent on Linux versions or kernel versions and not require the application of a kernel patch script.

An example of instructions for an RSUSB build can be found at https://github.com/IntelRealSense/librealsense/issues/9931#issuecomment-964289692

dmipx commented 1 year ago

Hi @schulz-bosch.

You can apply patch manually following set of patch files with version you need.

Later kernels (5+):

Later kernels, updated:

If you not use metadata, you can skip metadata related patches.

Example: for 5.19, all kernels fully support HID timestamp, no need for hid patch, all formats already upstreamed.

realsense-metadata-jammy-master.patch

realsense-powerlinefrequency-control-fix-jammy.patch

For kernels 6.0-6.4, skip powerline frequency and hid patches: realsense-metadata-jammy-hwe-6.2.patch

For kernel 6.5 and above, you do not need to patch anything.

You will see no "version" change in module as we do not modify it in manual patches, only with DKMS patches. You can verify "metadata" patch, for D455 camera, pid 0x0b5c, like that: modinfo uvcvideo | grep usb:v8086p0B5C Expected result: alias: usb:v8086p0B5Cd*dc*dsc*dp*ic0Eisc01ip00in*

MartyG-RealSense commented 1 year ago

Hi @schulz-bosch Do you require further assistance with this case, please? Thanks!

schulz-bosch commented 1 year ago

Hi @MartyG-RealSense, @dmipx , sorry for the late reply and thanks for the great explanation regarding the patch files! I think we are fine for now. The RSUSB backend is not an option for us, because we have a multi-camera setup and really depend on robustness / reliability.

MartyG-RealSense commented 1 year ago

Thanks very much @schulz-bosch for the update

This case should be kept open as it is associated with an internal Intel feature request for adding support for kernel 6.1, but you do not need to take any further action on this case. Thanks again!

Sylvania2 commented 1 year ago

@dmipx "For kernel 6.5 and above, you do not need to patch anything." does this mean that we can use kernel 6.5.0-1006-oem? I happened to update my kernel to kernel 6.2.0-36-generic by mistake and have since had problems with missing pointcloud as in issue 1234. What would be the best solution: downgrade to 5.19 and patch or use kernel 6.5.0-1006-oem?

Best regards

schulz-bosch commented 12 months ago

Hello again. Sorry for having another question.

@MartyG-RealSense : We are currently still using this library on version v2.50.0. For our 5.15 kernel however, we use the patch files from the master branch (I think, v2.50.0 didn't have jammy patch files at all). So I was wondering:

Thanks in advance!

MartyG-RealSense commented 12 months ago

Hi @schulz-bosch Before kernel 5.15 support was officially introduced, there was an unofficial Ubuntu 22.04 Jammy test patch for 5.15 made available in July 2022 at https://github.com/IntelRealSense/librealsense/issues/10439#issuecomment-1171965106

In July 2022 the latest SDK version was 2.50.0. The 2.51.1 SDK was introduced in August 2022, the.month after the unofficial patch's release.

The official 5.15 kernel patch support was introduced in SDK 2.53.1 onwards. You can avoid the need to patch the kernel though if you build the SDK from source code with CMake with the flag -DFORCE_RSUSB_BACKEND=TRUE included in the CMake build instruction, as an RSUSB = true build of the SDK is not dependent on Linux versions or kernel versions and does not require kernel patching (it bypasses the kernel).

schulz-bosch commented 12 months ago

Hi @MartyG-RealSense . As mentioned earlier already , rsusb is not an option for us. The question was, if there are compatibility issues if we use an older SDK (2.50.0) with a patched 5.15 kernel?

MartyG-RealSense commented 12 months ago

If your project does not require camera metadata (which kernel patching provides support for) then I would recommend trying to run the 2.50.0 SDK with an unpatched kernel. The SDK can still work with kernel versions that are not officially supported, though there may be unforseeable consequences in regards to stability. The best way to find out is to try it without patching and see how it performs.

schulz-bosch commented 11 months ago

I am running a 5.15 kernel with patches, and compared the 2.50.0 with the 2.54.2. I noticed that "rs-enumerate-devices -s" is very slow in version 2.54.2. I tested querying 2 devices, and it took 2.059930337s with 2.54.2, and 0.063554261s with 2.50.0. Is there any reason for that?

MartyG-RealSense commented 11 months ago

As you are using SDK 2.54.2, have you updated your camera firmware driver to version 5.15.1.0 please? This is the recommended firmware for 2.54.2, and using an older firmware version with it could result in errors.

schulz-bosch commented 11 months ago

Yes, the cameras are updated.

MartyG-RealSense commented 11 months ago

Thank you for the confirmation. I will check about your issue with @dmipx who is a kernel expert.

Hi @dmipx Are you able to replicate slowdown of rs-enumerate-devices in SDK 2.54.2 reported by @schulz-bosch at https://github.com/IntelRealSense/librealsense/issues/11745#issuecomment-1807853876 please? Thanks!

dmipx commented 11 months ago

I am running a 5.15 kernel with patches, and compared the 2.50.0 with the 2.54.2. I noticed that "rs-enumerate-devices -s" is very slow in version 2.54.2. I tested querying 2 devices, and it took 2.059930337s with 2.54.2, and 0.063554261s with 2.50.0. Is there any reason for that?

We are talking about same test on same device? The enumeration process polling all system video devices it can find in /dev, getting compatibility options. If there is many video devices in the system it can take too much time to complete. For case with 128 devices it can take 30 seconds.

schulz-bosch commented 11 months ago

@dmipx yes, same device, same kernel, same two realsense cameras.

Nir-Az commented 11 months ago

Hi @schulz-bosch , how do you use LibRS SDK? From Debians? Build from source? Any CMake flags that you change from the default values during build process?

schulz-bosch commented 11 months ago

@Nir-Az we build with cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TOOLS=true -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=true -DBUILD_WITH_TM2=false -DCHECK_FOR_UPDATES=false -DCMAKE_PREFIX_PATH=<prefix> -DCMAKE_INSTALL_PREFIX=<prefix> -DCMAKE_INSTALL_RPATH=<rpath> -DCMAKE_BUILD_WITH_INSTALL_RPATH=FALSE -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH -DIMPORT_DEPTH_CAM_FW=OFF -DBUILD_SHARED_LIBS=ON

important: we do not have libudev-dev installed as build dep, and we build libusb with "--disable-udev", because libudev is (afawk) GPL-2. the build environment and cmake flags have been identical in my tests.

Nir-Az commented 11 months ago

OK, that's an important input. We will have to investigate it but I cannot promise it will get a high priority as this is not breaking any KPI currently. But I must say it is interesting and we will take a look. Thanks for raising it.