ARM support and the necessary kernel patches

[Maghoumi]

I need to be able to use Jetson TX1 with R200. I've already seen #7 and #21 and I know that SSSE3 instructions need to be rewritten with NEON or CUDA.

Before attempting such a thing, I need to know what other modifications are required for using R200 with TX1. It seems that librealsense requires some kernel patches to work on regular x86 linux. Considering that the latest Linux4Tegra release still uses v3.10. I'm assuming I need to switch to upstream (or 4.4)?

What other patches/modifications are needed? Any suggested starting point?

Thanks :)

[ddiakopoulos]

Although it's not really as stable as the video4linux backend, it will be a lot easier for you to use the libuvc since it doesn't require any kernel patching. If it's a research project and you don't mind the occasional unplug/replug (setting expectations here!), then the libuvc backend will work with no modification (other than that SSSE3 function).

[Maghoumi]

Thank you @ddiakopoulos for your prompt response! Yes this will be for research purposes so as long as it runs, I'm ok with it! So assuming that I have replaced the SSSE3 instructions with NEON/CUDA, I can just compile with UVC backend and expect it to work? What about the kernel version? Can I conclude that v3.19 is fine and I don't need to update?

[ddiakopoulos]

@Maghoumi Kernel version is fine as-is with the libuvc backend, no update needed. There is a fork somewhere on Github where someone used some SSE2NEON header file to transliterate that function. Additionally, if you look back in the commit history, we had a naive yuy2=>rgb implementation that also worked.

[Maghoumi]

Awesome that's great news :+1: I fond the naive implementation in the commits... Will have to spend some time and figure out how it's working currently so that I can write the replacement for ARM.

Thanks again for your help!

[ddiakopoulos]

:+1:

[Maghoumi]

Guys! I can confirm that R200 works on Jetson TX1 with all streams enabled at 60 FPS :+1: :+1: Proof:

I reverted back to the naive YUY2 to RGB implementation in this commit. It turned out as a very dirty modification (I needed to a proof-of-concept thing fast to determine whether it works at all on the Jetson board). It's indeed very interesting to get 60 FPS on this thing even with the naive implementation. As suggested previously, I used the libuvc backend (and that unplug/replug is present :( ) If I ever find some free time, I'll write proper YUY2 to RGB implementations with proper conditional compilation blocks and will submit a pull request (there's a strong emphasis on "if").

Thanks and sorry for posting in a closed issue :)

[scott-eddy]

@Maghoumi how were you able to add the uvc kernel patch to your TX1? I have a TK1 that I would like to get an R200 up and running on but can't get the uvc patches to work. Trying to build 7eccebf without those uvc patches I get: lib/librealsense.so: undefined reference to rsimpl::uvc::get_pu_control_range(rsimpl::uvc::device const&, int, rs_option, int*, int*) collect2: error: ld returned 1 exit status Which looks like I have issues with linking uvc.
Full disclosure I am using the grinch 21.3.4 kernel so the output of uname -r is 3.10.40-grinch-21.3.4

Any advice?

[Maghoumi]

@scott-eddy As mentioned in the 2nd post in this issue, you don't need any kernel patches. You need to build the library with the LIBUVC backend. Use my fork (https://github.com/Maghoumi/librealsense.git). First install libuvc and then do make BACKEND=LIBUVC on the cloned fork.

[scott-eddy]

@Maghoumi Thanks for your fork, building with it works swimmingly!

[Maghoumi]

@scott-eddy You're welcome! Does that mean you got it running on your TK1?

[teknotus]

I've successfully gotten the TK1 working with the F200 using 3.10.40-grinch-21.3.4 as the basis. I even made a makefile that does most of the work.

https://github.com/teknotus/jetson-grinch-installer

I didn't get as far as the RealSense patch in that makefile, but it completely sets up the build environment, and does other important stuff like set the port to be USB 3.0 instead of the default 2.0. This was written in May of last year long before librealsense was released so I didn't test with it.

[scott-eddy]

@Maghoumi R200 working with TK1 and grinch kernel

[jvgomez]

@Maghoumi R200 working with TK1 ~40 FPS, your fork and default Linux4Tegra 21.4 kernel 3.10.40-gdacac96.

Any idea why I do not get 60FPS?

[Maghoumi]

Hey, Glad to hear that. Give me a couple of hours. I created another version but didn't push it to GitHub. That one ran faster than the current fork because of cleaner code. I will find it and push it to my fork.

Thanks On Apr 8, 2016 10:48 AM, "Javier V. Gómez" notifications@github.com wrote:

@Maghoumi https://github.com/Maghoumi R200 working with TK1 ~40 FPS, your fork and default Linux4Tegra 21.4 kernel 3.10.40-gdacac96.

Any idea why I do not get 60FPS?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/IntelRealSense/librealsense/issues/73#issuecomment-207461531

[jvgomez]

@Maghoumi thanks a lot! Are you considering updating with the commits in this fork?

I am wondering if somebody upgraded TK1 kernel to 4.4 and then used the Video4Linux backend.

[Maghoumi]

@jvgomez No problem. I will likely pull the latest stuff from librealsense's repo, add my changes and push to my own fork. I doubt librealsense's developers would agree to accept a PR because I'm still doing YUY2 to RGB conversion using for loops (rather than ARM NEON instructions). I haven't really had time to put into it and the for loops work for me for now : )

Don't know about TK1 but I tried using the upstream kernel (linux-next) on TX1. It booted but it was particularly useless as I couldn't get NVIDIA GPU drivers to work with it. I guess we need to wait for something from NVIDIA to try the V4L backend.

[Maghoumi]

@jvgomez Apologizes for the delay! We got stuck in the longest PhD defense in the history of mankind : D Anyways, please checkout the latest version of my fork (https://github.com/Maghoumi/librealsense) and see if that works any better on your TK1 (it still works at 60 FPS on my TX1 and works much faster than the previous one on my laptop).

If you don't see any meaningful improvements, let me know so that I can look into it : )

[jvgomez]

@Maghoumi thanks for the update!. First of all, I had to comment out these lines in the makefile:

UNAME_P := $(shell uname -p)
ifneq ($(filter arm%,$(UNAME_P)),)
        CXXFLAGS += -mssse3 
endif

or otherwise I was getting the error .g++:error: unrecognized command line option '-mssse3'

I cannot see too much improvement, but there might be other reasons (I am using an USB hub for instance). But setting task priority to very high it is stable around 60FPS (no idea about that before).

[Maghoumi]

@jvgomez I think that USB hub you're using might be causing the issues. I will take a look at the makefile again (strange that it compiled for me when I tried it... I compiled under TX1 itself).

[teknotus]

@jvgomez I think that USB hub you're using might be causing the issues.

I recommend getting a USB OTG adapter to use with the TK1 and putting all of your USB 2.0 or lower devices on that. Then you can connect the camera directly to the USB 3.0 port.

[jvgomez]

@teknotus That is actually a good point!

[sclin-loauas]

@Maghoumi Hello. I am trying to use R200 on tx1. I have installed your fork of librealsense for ARM. There is no error in compiling and installation. However, a error message shows up when I try to start the example.

RealSense error calling rs_create_context(api_version:4): uvc_set_ctrl(...) returned LIBUSB_ERROR_PIPE

I tried the examples with command sudo and saw the same error. Does this mean some error in the USB port?

[sclin-loauas]

The problem is solved. However I do not know why. I flash the system using JetPack and re-install the lib. It does not work at first. But It goes well after I unplug and plug in the camera several times.

[Maghoumi]

@sclin-loauas Yeah this is a known problem.... A while back I was looking at compiling correct kernel modules to get V4L backend to work... I didn't have much success. Take a look at the discussion here: https://github.com/Maghoumi/librealsense/pull/2 JetsonHacks has found a solution for TK1

[pczzy]

I will try my R200 ON TK1

[datlife]

Confirmed Working on my Jetson TK1 (Kernel: 3.10.40-ga7da876)

[jawad-tech]

I am building librealsense2 library in Jetson TX2 but unable to build it. I am following the

https://github.com/IntelRealSense/librealsense/blob/development/doc/distribution_linux.md https://github.com/IntelRealSense/librealsense/blob/development/doc/installation.md and try to run manually the ./patch-arch.sh and then cmake and make but it gives error related to hexfloat

" /home/nvidia/librealsense/librealsense/src/media/ros/ros_writer.cpp:296:52: error: ‘hexfloat’ is not a member of ‘std’ frame_timestamp_msg.value = to_string() << std::hexfloat << std::fixed << pose->get_frame_timestamp(); "

Then I am using the installation tutorials as https://github.com/jetsonhacks/buildLibrealsense2TX

but on make the file in the build folder, like after 21 percent it gives error as : " /home/nvidia/librealsense/librealsense/src/media/ros/ros_writer.cpp:296:52: error: ‘hexfloat’ is not a member of ‘std’ frame_timestamp_msg.value = to_string() << std::hexfloat << std::fixed << pose->get_frame_timestamp(); " Also I do not able to find sudo apt-get install librealsense2-dkms ???? but I found all libreasense2-dev or dbg libraries.

I am following this thread #2448 it also has the ssame error but i do not find the solution.

| Required Info | Camera Model is D415 | Firmware Version | Operating System & Version is UBUNTU 16.04 on Nvidia Jetson Tx2 with v4.4.38 | Kernel Version (Linux Only) is 4.4.38 | Platform is Nvidia Jetson TX2 (aarch64 system) g++ version is 4.9.4

Note: I have successfully build the library on my x86_64 pc but have problem on aarch64 system.

[jawad-tech]

https://github.com/Maghoumi/librealsense

Is this your support also available for D400 series

[nikhilkoditala]

https://github.com/Maghoumi/librealsense

@jawad-tech are you able to make this work with D400 series