search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.43k stars 4.8k forks source link

D435i doesn't work in Realsense-Viewer, but D435 does work. (Odroid) #3244

Closed johnbirk closed 5 years ago

johnbirk commented 5 years ago

| Camera Model | { D435 and D435i } | | Firmware Version | (D435i - FW 5.10.13) ; D435 - FW 5.10.6 | | Operating System & Version | {Ubuntu 16.04| | Kernel Version (Linux Only) | Linux Odroid 4.14.94-137 | | Platform | Odroid xu4 | | SDK Version | 2.18.1 } | | Language | {C} | | Segment | {Robot } |

When I use a D435 with realsense-viewer, it works. When I plug in a D435i it doesn’t work and generates the following message: Warning (3017527296) (ds5-factory.cpp:578) DS5 group_devices is empty. Realsense-viewer says "Connect a Realsense Camera or Add Source", but otherwise blank, except the note that librealsense version 2.18.1.

I found this: https://github.com/IntelRealSense/librealsense/commit/f6de6ea90e0f6491576073399d18f2f0c17a7180 which seems to imply a fix by 2.18.1. However, I am using 2.18.1.

Both cameras work on a Dell i7 laptop.

What to do?

dorodnic commented 5 years ago

Hello @johnbirk Are you building the SDK with FORCE_LIBUVC CMake flag?

johnbirk commented 5 years ago

I used the advice in https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md "cmake ../ -DBUILD_EXAMPLES=true - Builds librealsense along with the demos and tutorials"

According to: https://github.com/IntelRealSense/librealsense/wiki/Build-Configuration FORCE_LIBUVC "has well known limitations and is not recommended to be used in end-products." Are you recommending its use or trying to make sure I didn't use it because it might have caused the failure with the D435i?

dorodnic commented 5 years ago

I asked to make sure you are not using it for the D435i (it does not support IMU yet, but we are working on it to offer an alternative solution) With default build settings, the D435i should work with proper distributions based on kernel 4.14. Kernel patches are recommended but AFAIK not required (@ev-mp, will D435i work with inbox HID-IIO driver from 4.14?) Hardware timestamps will not be provided without applying kernel patches. jetsonhacks.com recently published a guide for patching NVidia specific kernel for D435i, something similar can be done for ODROID, but we do not officially support non-Intel platforms. Regardless, we will try to assist with troubleshooting. Perhaps the HID-IIO (IMU) driver was not loaded, or the camera was not detected. @ev-mp / @RealSenseCustomerSupport could you please assist?

ev-mp commented 5 years ago

@johnbirk , please provide the kernel log ($dmesg -wT) of connecting the camera to the PC and then running the realsense-viewer. Additionally, when you work with D435 on Odroid - is it recognized as USB2 org USB3 ? (rs-enumerate-devices)

johnbirk commented 5 years ago

The D435 is listed in realsense-viewer as USB3.2 Also in rs-enumerate-devices, the D435 is listed as USB 3.2 See attached file for the requested diagnostics D435iDiagnosticsOdroid.txt

RealSenseCustomerSupport commented 5 years ago

Hi @johnbirk,

This looks like a similar issue with #3165. Is the hid patch (./scripts/realsense-hid-ubuntu-${ubuntu_codename}-${kernel_branch}.patch) applied according to your Odroid platform. By the way, is it Odroid XU4 board? I will see if I can have one to have a further check.

johnbirk commented 5 years ago

The Odroid board is an XU4. During installation, the following patch from "Linux Ubuntu Installation" https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md was used: ./scripts/patch-realsense-ubuntu-odroid.sh Where are the instructions for the "hid" patch?

I found this patch ./scripts/patch-realsense-ubuntu-lts.sh https://github.com/IntelRealSense/librealsense/blob/master/scripts/patch-realsense-ubuntu-lts.sh in "Linux Ubuntu Installation" It took some time to run.

I didn't run this patch: ./scripts/patch-ubuntu-kernel-4.16.sh The instruction in "Linux Ubuntu Installation" implied that it was only applicable to 4.16, and the Odroid runs with a 4.14 kernel.

RealSenseCustomerSupport commented 5 years ago

Hi @johnbirk,

Basically, you can refer to below script to have the uvc patches. But, due to IIO module is not built as XU4 default kernel 4.14. You need to reconfigure the kernel (.config file),build and install the modules. https://github.com/IntelRealSense/librealsense/blob/master/scripts/patch-realsense-ubuntu-odroid-xu4-4.14.sh

To build the required HID/IIO modules, you can run script like below: LOCAL_VERSION=$(uname -r) bash scripts/config --file .config \

    --set-str LOCALVERSION $LOCAL_VERSION \
    --enable IIO_BUFFER \
    --module IIO_KFIFO_BUF \
    --module IIO_TRIGGERED_BUFFER \
    --enable IIO_TRIGGER \
    --set-val IIO_CONSUMERS_PER_TRIGGER 2 \
    --module HID_SENSOR_IIO_COMMON \
    --module HID_SENSOR_IIO_TRIGGER \
    --module HID_SENSOR_HUB \
    --module HID_SENSOR_ACCEL_3D \
    --module HID_SENSOR_GYRO_3D

Refer to below about XU4 kernel 4.14 build instructions. But, you need to patch hid (patch -p1 < ../scripts/realsense-hid-ubuntu-bionic-master.patch) https://wiki.odroid.com/odroid-xu4/software/building_kernel

I will modify/update above XU4-4.14 patch script in order to have a proper support for D435i on Odroid XU4.

johnbirk commented 5 years ago

I would appreciate a simple process to get the D435i working on the Odroid. As stated earlier, I applied ./scripts/patch-realsense-ubuntu-odroid.sh because it is mentioned in the "Linux Ubuntu Installation" instructions. I did not apply ./scripts/patch-realsense-ubuntu-odroid-xu4-4.14.sh. Didn't know about it. Should I apply this script in its current form in hopes of getting D435i to work? There is a statement above about modifying/updating the XU4-4.14 patch script. How will I get this? Will I find this modification in a future release of the SDK or can I get it without a complete reinstallation?

Here are some possibly relevant scripts I just found from the SDK: /Intel/librealsense-2.18.0/scripts/patch-hid-16.04.sh /Intel/librealsense-2.18.0/scripts/patch-realsense-ubuntu-lts.sh /Intel/librealsense-2.18.0/scripts/patch-realsense-ubuntu-odroid-xu4-4.14.sh /Intel/librealsense-2.18.0/scripts/patch-realsense-ubuntu-odroid.sh /Intel/librealsense-2.18.0/scripts/patch-ubuntu-kernel-4.16.sh It would be helpful to know the circumstances when each of these should be applied. Also, are they order dependent?

If you get a chance, it would also help to know what HID, IIO, AND UVC stand for.

RealSenseCustomerSupport commented 5 years ago

Hi @johnbirk,

Please try below script to have the correct kernel update and the required patches for D435i. Copy the script to your librealsense script folder and run "./script/patch-realsense-Ubuntu-odroid-xu4-4.14.sh". https://github.com/freemanlo/librealsense/blob/odroid_build_4.14/scripts/patch-realsense-ubuntu-odroid-xu4-4.14.sh HID (Human Interface Device) and UVC (USB Video Class) protocol used for the communication of D435i video and IMU streams.

johnbirk commented 5 years ago

I tried the latest edition of the github script recommended above. I copied it as a .txt and renamed it .sh in the file system. I ran this script as shown below. The message "#!/bin/bash: not found" was not encouraging. I tried the realsense-viewer, but still no luck with the D435i.

odroid@odroid:~/Intel/librealsense-2.18.1/scripts$ sudo ./patch-realsense-ubuntu-odroid-xu4-4.14.sh ./patch-realsense-ubuntu-odroid-xu4-4.14.sh: 1: ./patch-realsense-ubuntu-odroid-xu4-4.14.sh: #!/bin/bash: not found -e \e[32m Remove all RealSense cameras attached. Hit any key when ready./patch-realsense-ubuntu-odroid-xu4-4.14.sh: 18: read: arg count odroid@odroid:~/Intel/librealsense-2.18.1/scripts$

AfterComment - after this comment: Don't know why the terminal activity above was crossed out.

odroid@odroid:/bin$ realsense-viewer MESA-LOADER: failed to retrieve device information libGL error: unable to load driver: exynos_dri.so libGL error: driver pointer missing libGL error: failed to load driver: exynos 26/02 17:38:47,092 WARNING [3017596928] (ds5-factory.cpp:578) DS5 group_devices is empty. 26/02 17:38:48,080 WARNING [3017596928] (ds5-factory.cpp:578) DS5 group_devices is empty. 26/02 17:38:48,280 WARNING [3017596928] (ds5-factory.cpp:578) DS5 group_devices is empty. 26/02 17:38:48,482 WARNING [3017596928] (ds5-factory.cpp:578) DS5 group_devices is empty. 26/02 17:38:48,769 WARNING [3017596928] (ds5-factory.cpp:578) DS5 group_devices is empty.

I tried to download the script on github via vi. I just call the file “special”.

odroid@odroid:~/Intel/librealsense-2.18.1/scripts$ sudo ./special [sudo] password for odroid:

Remove all RealSense cameras attached. Hit any key when ready

./special: line 23: ./scripts/patch-utils.sh: No such file or directory odroid@odroid:~/Intel/librealsense-2.18.1/scripts$

This time the complaint was line 23: ./scripts/patch-utils.sh: No such file or directory

But "patch-utils.sh" exists! No luck again with realsense-viewer for D435i.

Please continue your help!

RealSenseCustomerSupport commented 5 years ago

Hi @johnbirk,

Please go to "~/Intel/librealsense-2.18.1" and run the script like below. It should run normally. odroid@odroid:~/Intel/librealsense-2.18.1$ sudo ./scripts/patch-realsense-ubuntu-odroid-xu4-4.14.sh

freemanlo commented 5 years ago

Make sure in /Intel/librealsense-2.18.1 and run below: ./script/patch-realsense-ubuntu-odroid-xu4-4.14.sh

johnbirk commented 5 years ago

I ran the script as requested. It produced a lot of text on the terminal. However, realsense-viewer still did not work for the D435i, although it still works for the D435. The attached file shows the terminal response. OdroidD435i.txt

RealSenseCustomerSupport commented 5 years ago

Hi @johnbirk,

From the log, it looks you ran the old script, please make sure to replace it with below new script. https://github.com/freemanlo/librealsense/blob/odroid_build_4.14/scripts/patch-realsense-ubuntu-odroid-xu4-4.14.sh

johnbirk commented 5 years ago

The old script was not replaced. It has now been replaced as advised. It ran for 25 minutes and ended on a positive note "Patched kernels modules were created successfully" During the 25 minutes it produced hundreds, if not thousands of lines of output, some of which is included in the attached file:

OdroidD435iScriptOutput.txt

The last output of the script was "Script has completed. Please consult the installation guide for further instruction." What installation guide are they referring to? Is it "Linux Ubuntu Installation"?

Unfortunately, the D435i still does not run on realsense-viewer on Odroid xu4.

RealSenseCustomerSupport commented 5 years ago

Hi @johnbirk,

Yes, refer to "Linux Ubuntu Installation" and check if you install Intel Realsense permission scripts located in librealsense source directory: sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/ sudo udevadm control --reload-rules && udevadm trigger Remember to reboot your system. And, then try realsense-viewer. If it still doesn't work, show us the log. You can configure the "Settings" -> General" in realsense-viewer to adjust the minimal log severity to Debug.

johnbirk commented 5 years ago

Success! The realsense-viewer appeared for the D435i camera on an xu4 Odroid with Ubuntu.

At first the motion module worked, but when the motion module was turned off and turned on again, it didn't work. The motion module worked again and didn't work again, when the realsense-viewer was started again. See details further below:

Since the realsense-viewer is working, the original problem of this ticket has been resolved. Should this ticket be closed and another ticket started as needed to address other problems?

Error message in the realsense-viewer upon turning the motion module on for a second time:

Backend in rs2_stop(sensor:0xabf090f0): Failed to enable_sensor /sys/devices/platform/soc/soc:usb3-0/12000000.dwc3/xhci-hcd.3.auto/usb4/4-1/4-1.1/4-1.1:1.5/0003:8086:0B3A.0006/HID-SENSOR-2000e1.10.auto/enable_sensor Last Error: Permission denied

Activity on the terminal (Don't know what causes cross outs):

odroid@odroid:~$ realsense-viewer MESA-LOADER: failed to retrieve device information libGL error: unable to load driver: exynos_dri.so libGL error: driver pointer missing libGL error: failed to load driver: exynos 27/02 17:17:39,781 WARNING [2843054656] (types.cpp:57) hwmon command 0x4f failed. Error type: No data to return (-21). 27/02 17:17:39,930 WARNING [2843054656] (sensor.cpp:338) Unregistered Media formats : [ UYVY ]; Supported: [ ] 27/02 17:18:27,310 ERROR [3016867840] (types.h:214) Failed to enable_sensor /sys/devices/platform/soc/soc:usb3-0/12000000.dwc3/xhci-hcd.3.auto/usb4/4-1/4-1.1/4-1.1:1.5/0003:8086:0B3A.0006/HID-SENSOR-2000e1.10.auto/enable_sensor Last Error: Permission denied 27/02 17:22:01,340 WARNING [3016867840] (types.cpp:57) open(...) failed. Hid device is streaming! 27/02 17:22:12,337 WARNING [3016867840] (types.cpp:57) open(...) failed. Hid device is streaming! 27/02 17:22:36,654 WARNING [3016867840] (types.cpp:57) open(...) failed. Hid device is streaming! 27/02 17:23:00,877 WARNING [3016867840] (types.cpp:57) open(...) failed. Hid device is streaming! 27/02 17:23:49,570 WARNING [3016867840] (types.cpp:57) open(...) failed. Hid device is streaming! 27/02 17:24:40,132 ERROR [3016867840] (types.h:214) Failed to enable_sensor /sys/devices/platform/soc/soc:usb3-0/12000000.dwc3/xhci-hcd.3.auto/usb4/4-1/4-1.1/4-1.1:1.5/0003:8086:0B3A.0006/HID-SENSOR-2000e1.10.auto/enable_sensor Last Error: Permission denied 27/02 17:24:40,132 ERROR [3016867840] (sensor.cpp:778) An error has occurred while stop_streaming()! odroid@odroid:~$

RealSenseCustomerSupport commented 5 years ago

Hi @johnbirk,

Please close this and open another ticket with logs to track the new problem.