search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.49k stars 4.8k forks source link

Issues while using two realsense #468

Closed xiechangwu closed 6 years ago

xiechangwu commented 7 years ago
Required Info
Camera Model R200
Firmware Version R200={1.0.72.06} SR300={3.17.0.0}
Operating System & Version Ubuntu 16.04
Kernel Version (Linux Only) 4.4 for PC/3.10.96 for TX1 with uvc patch
Build System Makefile
  1. I'm using two realsense on my board(nvidia TX1), an R200 and an SR300. Open the librealsense exmaple cpp-multicam, the default resolution is 640X480, averything goes will.
  2. In my application, we want use the 1080P resolution, so I change it and rebuild cpp-multicam.cpp(RGB:1080P, Depth:640x480)
  3. At this time, the R200 RGB data occures incorrect。
  4. The images seems like some pixel lines repeatlly covers other lines.

Left is the image for R200, and right for SR300. 1

Summary:

  1. I tried using above two camera with my PC(thinkPad with two standalone USB3.0 port), this problem also not disappeared, but the R200 image is better than on TX1 board.
  2. While I using two R200 with 1080P, this problem is the same.
  3. While I using only one R200 with 1080P plus 640X480 depth on TX1 board, the R200 RGB data sometimes not correct, but much better than two realsense.
  4. I've confirmed that the source image from librealsense is incorrect by saving the date after read date .
  5. TX1 board kernel version 3.10.96 with UVC module pached
  6. PC kernel version is 4.4 and installed with official guides

Source code what I've changed: //dev->enable_stream(rs::stream::depth, rs::preset::best_quality); //dev->enable_stream(rs::stream::color, rs::preset::best_quality); TO dev->enable_stream(rs::stream::depth, 640, 480, rs::format::z16, 30); dev->enable_stream(rs::stream::color, 1920, 1080, rs::format::rgb8, 30);

dorodnic commented 7 years ago

This sounds like side-effects of USB buffer overflows. It happens whenever the host PC is not pulling USB buffers from the device fast enough. At the moment, librealsense just passes such partial frames to the application. This can be verified by printing out the value of buf.bytesused at uvc-v4l2.cpp:344. During normal streaming this value should equal to _buffers[buf.index].length, but during overflows it would be lower. The overflows themselves are either caused by high CPU usage in parallel to streaming, or just a weak CPU (perhaps CPU is being thermal-throttled? In this case cooling can have big effect) As a possible work-around I have previously recommended to increase the number of URBs in uvcvideo from 5 to about 8 (change #define UVC_URBS 5 in /drivers/media/usb/uvc/uvcvideo.h, rebuild and replace the Kernel module). In theory this should increase the queue size between USB layer and UVC driver, reducing partial frames at expense of some extra latency. However, this might have other unexpected side-effects and would not be recommended in production environment.

P.S. Just to make sure, you are using V4L back-end and not LIBUVC, correct?

xiechangwu commented 7 years ago

Hi, dorodnic. Thanks very much for you help.

rs.warn: Subdevice 1 frame status 0x1 rs.warn: buf.bytesused=3932160 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=4147200 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=589064 sub->buffers[buf.index].length:589064 rs.warn: Subdevice 1 frame status 0x1 rs.warn: buf.bytesused=3953664 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=589064 sub->buffers[buf.index].length:589064 rs.warn: Subdevice 1 frame status 0x1 rs.warn: buf.bytesused=3910656 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=3459072 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=589064 sub->buffers[buf.index].length:589064 rs.warn: Subdevice 1 frame status 0x1 rs.warn: buf.bytesused=3867648 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=3437568 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=4147200 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=614400 sub->buffers[buf.index].length:614400 rs.warn: buf.bytesused=4147200 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=3932160 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=614400 sub->buffers[buf.index].length:614400 rs.warn: buf.bytesused=4147200 sub->buffers[buf.index].length:4147200 rs.warn: buf.bytesused=589064 sub->buffers[buf.index].length:589064

@dorodnic

xiechangwu commented 7 years ago

@dorodnic Please note that this issue only appears for R200， the SR300 doesn't has this problem. While I using two SR300 for image capture, this problem also disappeared. Is it caused by an additional picture called "infrared" when using R200?

dorodnic commented 7 years ago

If you are enabling Infrared and Infrared2 on the R200 in your application, this would be the first thing to disable. However, it might be that your target platform simply struggles to handle two 1080P streams + depth data. What is the CPU utilization when the problem is happening? When you are requesting RGB data from librealsense, we are internally doing YUY to RGB conversion, and it is known to be very CPU intensive. Even if SSE instructions are available and enabled, this might be too much for some hosts to handle and keep up with the USB transfer. Could you try your test, but ask for YUY format instead of RGB in the sample App? (You could consider offloading YUY to RGB conversion to the GPU. We are not doing any of that inside librealsense to avoid adding OpenCL as a dependency, but I know it has been done before)

xiechangwu commented 7 years ago

@dorodnic

Any suggestions? Thanks!

jlgarcia75 commented 6 years ago

For best support, post issues in the RealSense Community.

Hello,

Do you still need help with this issue?

Best regards, Jesus Intel Customer Support