Jetson TX2 D435 high resolutions available?

[alex-precosky]

Required Info
Camera Model	D435
Firmware Version	5.10.6
Operating System & Version	Ubuntu 16.04.5 LTS
Kernel Version (Linux Only)	4.4.38-28.2.86.0
Platform	NVIDIA Jetson TX2
SDK Version	V4L2
Language	C
Segment	Robot

Issue Description

Sorry if this is covered elsewhere. I searched through online documentation and old issues.

Kernel version as above, but I've also the following two kernel patches applied: realsense-metadata-ubuntu-xenial-master.patch realsense-camera-formats_ubuntu-xenial-master.patch

I'm attempting to use V4L2 to acquire at the highest resolution, 1920x1080 @ 30fps using yavta (https://github.com/fastr/yavta) from the /dev/video* device representing the colour image. Up to 964x540@60 I'm able to record proper images. Anything higher in resolution though and I seem to start getting partial images, i.e. they only seem to have a few rows of pixels in them. The yavta tool even warns me of this. Even at lower frame rate, i.e. 6 fps, this still occurs at the higher resolutions. I'll paste some sample output below:

Is this something I might expect to be resolved if I use librealsense in an application instead of just using a stock V4L2 recording application? If it would help answer this question, I'll get realsense-viewer compiled for my board and try with that to see if I can get this resolution in there. Thought I would ask here first though in case someone knows right away any obvious issue with my setup.

yavta git:(master) ✗ ./yavta --capture=10 --size 1920x1080 -t 1/30  /dev/dex-cam-color-front_center -Fimage -fYUYV
Device /dev/dex-cam-color-front_center opened: Intel(R) RealSense(TM) 435 (usb-0000:03:00.0-1).
Video format set: width: 1920 height: 1080 buffer size: 4147200
Video format: YUYV (56595559) 1920x1080
Current frame rate: 1/30
Setting frame rate to: 1/30
Frame rate set: 1/30
8 buffers requested.
length: 4147200 offset: 0
Buffer 0 mapped at address 0x7f93d15000.
length: 4147200 offset: 4149248
Buffer 1 mapped at address 0x7f93920000.
length: 4147200 offset: 8298496
Buffer 2 mapped at address 0x7f9352b000.
length: 4147200 offset: 12447744
Buffer 3 mapped at address 0x7f93136000.
length: 4147200 offset: 16596992
Buffer 4 mapped at address 0x7f92d41000.
length: 4147200 offset: 20746240
Buffer 5 mapped at address 0x7f9294c000.
length: 4147200 offset: 24895488
Buffer 6 mapped at address 0x7f92557000.
length: 4147200 offset: 29044736
Buffer 7 mapped at address 0x7f92162000.
0 (0) [-] 0 4147200 bytes 3137.296213 1541098780.600675
Warning: bytes used 65784 != image size 4147200
1 (1) [-] 1 65784 bytes 3137.316530 1541098780.625953
2 (2) [-] 2 4147200 bytes 3137.363352 1541098780.652882
Warning: bytes used 65784 != image size 4147200
3 (3) [-] 3 65784 bytes 3137.369404 1541098780.662457
4 (4) [-] 4 4147200 bytes 3137.430586 1541098780.730928
Warning: bytes used 65784 != image size 4147200
5 (5) [-] 5 65784 bytes 3137.447911 1541098780.763492
6 (6) [-] 6 4147200 bytes 3137.497735 1541098780.786864
Warning: bytes used 65784 != image size 4147200
7 (7) [-] 7 65784 bytes 3137.503783 1541098780.796394
8 (0) [-] 8 4147200 bytes 3137.564979 1541098780.865282
Warning: bytes used 65784 != image size 4147200
9 (1) [-] 9 65784 bytes 3137.582268 1541098780.903634
Captured 9 frames in 0.303648 seconds (29.639583 fps, 69372826.430604 B/s).
8 buffers released.

[alex-precosky]

Forgot to add, I'm using this USB 3.0 PCIe card http://www.vantecusa.com/products_detail.php?p_id=119

[dorodnic]

Hi @alex-precosky Well, on Linux librealsense is just a fancy wrapper on top of V4L2 really. So yavta is as good a tool as any for the job. In fact, we have the same check in backend-v4l2.cpp:959-968 What about the built-in USB3 port on the Jetson? From my experience with the Jetson, it should be absolutely capable of handling 1080p. There might be a lost frame here and there, but nothing major. Also, I might be saying something stupid (being Jetson newbie and all) but did you try it with jetson_clocks.sh?

[alex-precosky]

Cool thx for the info! Helpful to know it should indeed be possible to get that resolution.

I've tried the built-in USB3 port but the output is the same, in fact the exact same number of bytes are missing from the image when I use the built-in port. Yep I've run jetson_clocks.sh

I've only applied the two kernel patches listed in my first message, but maybe I need to patch using the approved method. Looks like some USB stuff in there. I'll see what I can do.

[dorodnic]

I think you can safely rule out kernel patches. The RGB sensor on the D400 / SR300 should behave 100% as a UVC compliant web-cam, with no extra steps. The only unusual thing about these cameras is that they work in Bulk USB mode and not Isochronous. Later is more widespread among web-cams. This can be related to what you are seeing if there is another high bandwidth device on the same USB controller (either another camera or maybe something like external drive)

[alex-precosky]

Ah gotchya. Just in case, I'm going to try taking the stock L4T kernel and applying the recommended patches and see if that makes a difference, since I might have other weird things set up in the kernel I was using. I'll keep this ticket updated with whatever I find

[alex-precosky]

Sadly no luck still. I tried a bunch of different D435 cameras just to make sure, and a few different Jetson TX2 boards. And tried building a kernel using https://github.com/jetsonhacks/buildJetsonTX2Kernel.git and booting off that on a stock Tegra JetPack system image but the problem remained. I'll see if Nvidia has any ideas. For my application, I only need 15 fps so I'll just take the 30 fps image stream and throw away bad frames as necessary.

[alex-precosky]

As another data point on this issue, I tried a different 1080p 30 fps YUYV output USB3 camera on this platform, and the frames are all complete without anything missing, and they appear fine when I inspect the images after acquiring them with yavta:

Device /dev/video3 opened: e-con's CX3 RDK with OV5640 (usb-0000:04:00.0-1).
Video format set: width: 1920 height: 1080 buffer size: 4147200
Video format: YUYV (56595559) 1920x1080
Current frame rate: 1/30
Setting frame rate to: 1/30
Frame rate set: 1/30
8 buffers requested.
length: 4147200 offset: 0
Buffer 0 mapped at address 0x7faf341000.
length: 4147200 offset: 4149248
Buffer 1 mapped at address 0x7faef4c000.
length: 4147200 offset: 8298496
Buffer 2 mapped at address 0x7faeb57000.
length: 4147200 offset: 12447744
Buffer 3 mapped at address 0x7fae762000.
length: 4147200 offset: 16596992
Buffer 4 mapped at address 0x7fae36d000.
length: 4147200 offset: 20746240
Buffer 5 mapped at address 0x7fadf78000.
length: 4147200 offset: 24895488
Buffer 6 mapped at address 0x7fadb83000.
length: 4147200 offset: 29044736
Buffer 7 mapped at address 0x7fad78e000.
0 (0) [-] 0 4147200 bytes 18665.700342 1543367715.818434
1 (1) [-] 1 4147200 bytes 18665.733670 1543367715.851539
2 (2) [-] 2 4147200 bytes 18665.766974 1543367715.884921
3 (3) [-] 3 4147200 bytes 18665.800321 1543367715.918279
4 (4) [-] 4 4147200 bytes 18665.833653 1543367715.951602
5 (5) [-] 5 4147200 bytes 18665.866985 1543367715.984937
..

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment] Hi alex-precosky,

Wonder if there is any update from you? Also any info from nvidia side as you wanted to check? I managed to repeat what you mentioned off a TX2 (w L$T 4.4.38 28.2), and got similar issue of not being able to capture all the frames. I will try to dig into it and see if I can find out more.

Thanks!

[RealSenseCustomerSupport]

---

Hi alex-precosky,

Wonder if there is any update on this one?

Here are some comments: As for a sanity check, there is no issue off an Intel NUC i5 with Ubuntu 16.04 when doing same yavta tests. D435 supports YUYV 1920x1080 30fps (check with v4l2-ctl command), and uses USB Bulk mode (lsusb info). To my understanding, it would indicate that there would some improvement outside RealSense SDK (comparing NUC i5 vs. TX2), and should have nothing to do with RealSense optimization.

Thanks!

[RealSenseCustomerSupport]

---

Hi alex-precosky,

Is there still anything needing help? Otherwise, this one will be closed soon.

Thanks!

[alex-precosky]

Hi thanks for the reply. I think I'm fine - I've worked around the issue by filtering out the partial frames and setting the frame rate higher than I need, and this seems fine.