Stream RAW10 from left and right imagers on D415

[mprat]

---

Required Info
Camera Model	D415
Firmware Version	5.10
Operating System & Version	Ubuntu 16.04
Kernel Version (Linux Only)	4.4.38-tegra
Platform	Jetson TX2
SDK Version	2.16.0
Language	C++

Issue Description

I would like to stream the RAW10 1920x1080 (i.e. full resolution) of the left and right imagers on my D415 camera, but I have not found the magic set of parameters to get this to work. I would expect this to work:

rs2::config cfg;
cfg.enable_stream(RS2_STREAM_INFRARED, 1, 1920, 1080, RS2_FORMAT_RAW10, 15);

rs2::pipeline _pipe;
_pipe.start(cfg);

... later ...

rs2::frameset data = _pipe.wait_for_frames();
rs2::frame infrared_1 = data.get_infrared_frame(1);

But when running my program in RS2_LOG_SEVERITY_DEBUG I see this output, which suggests that RAW10 streaming is not available:

 05/10 15:23:27,816 INFO [547938254848] (environment.cpp:85) Found 433 unreachable streams, 2165 extrinsics deleted
 05/10 15:23:27,826 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format 16-bit Depth data
 05/10 15:23:27,826 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format 16-bit Depth data
 05/10 15:23:27,827 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format 8-bit Greyscale
 05/10 15:23:27,827 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format UYVY 4:2:2
 05/10 15:23:27,827 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format 8-bit Greyscale
 05/10 15:23:27,827 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format 8-bit Greyscale L/R interleaved
 05/10 15:23:27,827 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format 12-bit Grey L/R interleaved
 05/10 15:23:27,847 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format YUYV 4:2:2
 05/10 15:23:27,847 DEBUG [547938254848] (backend-v4l2.cpp:1258) Recognized pixel-format 16-bit Bayer BGBG/GRGR (Exp.)
terminate called after throwing an instance of 'rs2::error'
  what():  Couldn't resolve requests
Aborted (core dumped)

Is streaming in RAW10 a supported feature? Am I missing some kind of driver?

[MartyG-RealSense]

Earlier in 2018, Intel support team member Jesus G said:

---

The RAW format is not exposed to end users. To view all supported stream formats for each sensor use the command:

rs-enumerate-devices -m

---

Y16 is an unrectified IR format you might consider as an alternative, though it only has a couple of supported resolutions as it is intended for camera calibration rather than everyday use in applications.

https://communities.intel.com/message/557408#557408

[mprat]

Thanks for the link, @MartyG-RealSense.

So my understanding is that there is basically no way to compare our own stereo algorithms with the output of the D4 (based on some custom quality metric) because we can't get access to the RAW frames from the left and right imagers? Is that correct?

[MartyG-RealSense]

There is some scripting for accessing the left and right IR imagers, though I'm not sure it's what you are seeking. A user who tried it said "It really is the raw data"

https://communities.intel.com/message/551304#551304

[dorodnic]

Hi @mprat What do you mean by RAW? You can get raw, unrectified (but also uncalibrated) images from the two infrared images by configuring the camera to Y16 format. There is some basic scaling going on to make the image visible when rendering as 16-bpp, but you could disable it if you want the raw 12-bpp data from the sensor.

[mprat]

@dorodnic I was assuming that because the left / right imagers were actually 10-bit RGB sensors that somehow all 3 10-bit RGB values were exposed to the end user - that's what I meant by RAW. When you stream the Y16 from the left / right imagers you only seem to get a single channel (luminance) rather than 3 RGB values.

[dorodnic]

I see. Yes, you are correct about the sensor being RGB. This can be done (I believe this format is referred to as W10 at the firmware level), but I'm not sure if latest D415 firmware exposes it or not. I did something similar on an engineering model. If you are interested, you can try the following patch and change the PID to RS415_PID. We can't guaranty that it will work (as this format is not part of the product data-sheet) but if it does, it seems like what you are looking for.

[mprat]

Ah, cool! Does that format only expose it for one imager stream ({RS2_INFRARED_STREAM, 1})? I could probably add another line in that patch to get it from the other one as well.

I'll try this and report back soon. Thanks for the pointer.

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment] Hi mprat,

How about your test result, if your question has been resolved, we'd like to close this ticket. Thanks!

[mprat]

I haven't tried it, but figured out a way to not need those images, so I'll close the issue. Thanks.