search

IntelRealSense / realsense_mipi_platform_driver

Intel® RealSense™ camera driver for GMSL* interface
GNU General Public License v2.0
14 stars 10 forks source link

readme

Intel® RealSense™ camera driver for GMSL* interface

D457 MIPI on Jetson AGX Xavier

The RealSense MIPI platform driver enables the user to control and stream RealSense 3D MIPI cameras. The system shall include:

Note: This MIPI reference driver is based on RealSense de-serialize board. For other de-serialize boards, modification might be needed.

image

Jetson AGX Xavier board setup

Please follow the instruction to flash JetPack to the Jetson AGX Xavier with NVIDIA SDK Manager or other methods NVIDIA provides. Make sure the board is ready to use.

Build kernel, dtb and D457 driver

The developers can set up the source code with NVIDIA's Jetson git repositories by using the provided setup script:

# Using setup script, recommended for developers. If JetPack version is not given, default version will be chosen.
./setup_workspace.sh [JetPack_version]

Or download Jetson Linux source code tarball from 

# JetPack 6.0
mkdir -p l4t-gcc/6.0
cd ./l4t-gcc/6.0
wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v2.0/toolchain/aarch64--glibc--stable-2022.08-1.tar.bz2 -O aarch64--glibc--stable-final.tar.bz2
tar xf aarch64--glibc--stable-final.tar.bz2 --strip-components 1
cd ../..
wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v2.0/sources/public_sources.tbz2
tar xjf public_sources.tbz2
cd Linux_for_Tegra/source
tar xjf kernel_src.tbz2
tar xjf kernel_oot_modules_src.tbz2
tar xjf nvidia_kernel_display_driver_source.tbz2

# JetPack 5.1.2
mkdir -p l4t-gcc/5.1.2
cd ./l4t-gcc/5.1.2
wget https://developer.nvidia.com/embedded/jetson-linux/bootlin-toolchain-gcc-93 -O aarch64--glibc--stable-final.tar.gz
tar xf aarch64--glibc--stable-final.tar.gz
cd ../..
wget https://developer.nvidia.com/downloads/embedded/l4t/r35_release_v4.1/sources/public_sources.tbz2
tar xjf public_sources.tbz2
cd Linux_for_Tegra/source/public
tar xjf kernel_src.tbz2

# JetPack 5.0.2
mkdir -p l4t-gcc/5.0.2
cd ./l4t-gcc/5.0.2
wget https://developer.nvidia.com/embedded/jetson-linux/bootlin-toolchain-gcc-93 -O aarch64--glibc--stable-final.tar.gz
tar xf aarch64--glibc--stable-final.tar.gz --strip-components 1
cd ../..
wget https://developer.nvidia.com/embedded/l4t/r35_release_v1.0/sources/public_sources.tbz2
tar xjf public_sources.tbz2
cd Linux_for_Tegra/source/public
tar xjf kernel_src.tbz2

# JetPack 4.6.1
mkdir -p l4t-gcc/4.6.1
cd ./l4t-gcc/4.6.1
wget http://releases.linaro.org/components/toolchain/binaries/7.3-2018.05/aarch64-linux-gnu/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz
tar xf gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz --strip-components 1
cd ../..
wget https://developer.nvidia.com/embedded/l4t/r32_release_v7.1/sources/t186/public_sources.tbz2
tar xjf public_sources.tbz2
cd Linux_for_Tegra/source/public
tar xjf kernel_src.tbz2

Apply D457 patches and build the kernel image, dtb and D457 driver.

# if using setup script
./apply_patches.sh [--one-cam | --dual-cam] apply [JetPack_version]

# or, if using direct download method
# ./apply_patches_ext.sh [--one-cam | --dual-cam] ./Linux_for_tegra/source/public [JetPack_version]
# for JP6.0, source path is ./Linux_for_tegra/source :
# ./apply_patches_ext.sh ./Linux_for_tegra/source 6.0

Note: The `--one-cam` and `--dual-cam` option applies only for JetPack 5.0.2,
compatible with adapter: https://store.intelrealsense.com/buy-intel-realsense-des457.html.
By setting the `--one-cam` option it builds DT with only camera on GMSL link A (default).
By setting the `--dual-cam` option it builds DT with dual cameras on GMSL link A and B.
The default is to single camera configuration for JetPack 5.0.2.

# build kernel, dtb and D457 driver
# install dependencies
sudo apt install build-essential bc flex bison
# method 1: build kernel Debian packages
./build_all_deb.sh [--no-dbg-pkg] [JetPack_version] [JetPack_source_dir]
# method 2: build kernel and modules only
./build_all.sh [JetPack_version] [JetPack_source_dir]

# remove our patches from JetPack kernel source code if using setup script
# ./apply_patches.sh reset [JetPack_version]

Debian packages will be generated in images folder.

Install kernel and D457 driver to Jetson Orin

JP6.0 build results - kernel image (not modified): `images/6.0/rootfs/boot/Image` - dtb overlay: `images/6.0/rootfs/boot/tegra234-camera-d4xx-overlay.dtbo` - oot modules: `images/6.0/rootfs/lib/modules/5.15.122-tegra/extra` Following steps required: 1. Copy entire directory `images/6.0/rootfs/lib/modules/5.15.122-tegra/extra/` from host to `/lib/modules/5.15.122-tegra/extra/` on Orin 2. Copy `tegra234-camera-d4xx-overlay.dtbo` from host to `/boot/tegra234-camera-d4xx-overlay.dtbo` on Orin 3. Run $ `sudo /opt/nvidia/jetson-io/jetson-io.py` 1. Configure Jetson AGX CSI Connector 2. Configure for compatible hardware 3. Jetson RealSense Camera D457 4. $ `sudo depmod` 5. $ `echo "d4xx" | sudo tee /etc/modules-load.d/d4xx.conf` 4. Reboot Copy them to the right places: ``` scp -r images/6.0/rootfs/boot/tegra234-camera-d4xx-overlay.dtbo nvidia@10.0.0.116:~/ scp -r images/6.0/rootfs/lib/modules/5.15.122-tegra/extra nvidia@10.0.0.116:~/ # RealSense metadata patched kernel modules scp -r images/6.0/rootfs/lib/modules/5.15.122-tegra/kernel/drivers/media/v4l2-core/videodev.ko nvidia@10.0.0.116:~/ scp -r images/6.0/rootfs/lib/modules/5.15.122-tegra/kernel/drivers/media/usb/uvc/uvcvideo.ko nvidia@10.0.0.116:~/ # Kernel Image with SENSOR_HID support for RealSense USB cameras with IMU scp -r images/6.0/rootfs/boot/Image nvidia@10.0.0.116:~/ ``` on target: ``` sudo cp ~/tegra234-camera-d4xx-overlay.dtbo /boot/ # backup: sudo tar -cjf /lib/modules/$(uname -r)/modules_$(uname -r)_extra.tar.bz2 /lib/modules/$(uname -r)/extra sudo cp -r ~/extra /lib/modules/$(uname -r)/ # enable RealSense metadata: sudo cp uvcvideo.ko /lib/modules/5.15.122-tegra/kernel/drivers/media/usb/uvc/uvcvideo.ko sudo cp videodev.ko /lib/modules/5.15.122-tegra/kernel/drivers/media/v4l2-core/videodev.ko # backup kernel (better to have additional boot entry in extlinux.conf) sudo cp /boot/Image /boot/Image.orig sudo cp Image /boot/Image ``` Enable d4xx overlay: With Jetson-IO tool: `sudo /opt/nvidia/jetson-io/jetson-io.py` 1. Configure Jetson AGX CSI Connector 2. Configure for compatible hardware 3. Jetson RealSense Camera D457 With command line `sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="Jetson RealSense Camera D457"` Expected: ``` nvidia@ubuntu:~$ sudo /opt/nvidia/jetson-io/config-by-hardware.py -n 2="Jetson RealSense Camera D457" Configuration saved to /boot/tegra234-camera-d4xx-overlay.dtbo. Reboot system to reconfigure. ``` Enable d4xx autoload: `echo "d4xx" | sudo tee /etc/modules-load.d/d4xx.conf` `sudo depmod` ---

Install kernel and D457 driver to Jetson AGX Xavier

  1. Install the kernel and modules

1.1 If building with build_all_deb.sh

Copy the Debian package linux-image-5.10.104-d457_5.10.104-d457-1_arm64.deb to the Jetson AGX Xavier board and install with sudo dpkg -i linux-image-5.10.104-d457_5.10.104-d457-1_arm64.deb. The header, libc-dev, dbg and firmware packages are optional.

1.2 If building with build_all.sh

The necessary files are:

And copy them to the right places:

sudo cp Image /boot
sudo cp tegra194-p2888-0001-p2822-0000.dtb /boot/dtb
sudo cp d4xx.ko /lib/modules/5.10.104-tegra/kernel/drivers/media/i2c/
sudo cp uvcvideo.ko /lib/modules/5.10.104-tegra/kernel/drivers/media/usb/uvc/
sudo cp videobuf-core.ko /lib/modules/5.10.104-tegra/kernel/drivers/media/v4l2-core/
sudo cp videobuf-vmalloc.ko /lib/modules/5.10.104-tegra/kernel/drivers/media/v4l2-core/
sudo depmod
  1. Edit /boot/extlinux/extlinux.conf primary boot option's LINUX/FDT lines to use built kernel image and dtb file:
LINUX /boot/Image-5.10.104-d457
FDT /usr/lib/linux-image-5.10.104-d457/tegra194-p2888-0001-p2822-0000.dtb
  1. Make D457 I2C module autoload at boot time: echo "d4xx" | sudo tee /etc/modules-load.d/d4xx.conf

After rebooting Jetson, the D457 driver should work.

NOTE