Announcement: Introducing the RealSense D457 GMSL/FAKRA stereo camera

[MartyG-RealSense]

Hi everyone,

Intel have introduced a new member of the RealSense stereo camera family called the D457. It is based around the D450 depth module and the new Vision Processor D4 V5.

https://www.intelrealsense.com/depth-camera-d457/

https://www.intel.com/content/www/us/en/newsroom/news/new-realsense-depth-camera-d457.html

Its listing in the official RealSense Store can be found here:

https://store.intelrealsense.com/buy-intel-realsense-depth-camera-d457.html

To quote the store page:

"The Intel® RealSense™ Depth Camera D457 is our first GMSL/FAKRA high bandwidth stereo camera. The D457 has an IP65 grade enclosure protecting it from dust ingress and projected water. D457 has an on-board Maxim serializer chip. A Maxim de-serializer is needed at host side to decode the streams".

---

Connect Tech provide support for D457 with their hardware products.

https://connecttech.com/connect-tech-offers-built-in-support-on-the-newly-announced-intel-gmsl-realsense-depth-camera-d457/

---

The RealSense Store retails an 8-pack of the D450 camera module and Vision Processor D4 V5 here:

https://store.intelrealsense.com/intelr-realsensetm-depth-module-d450-vision-processor-d4-board-v5-gmsl-fakra-8-pack.html

---

The dedicated data sheet document for the D457 can be downloaded here:

https://dev.intelrealsense.com/docs/intel-realsense-d400-series-product-family-datasheet

The official Intel CAD file for the D457 is here:

https://dev.intelrealsense.com/docs/stereo-depth-camera-d400#cad-files

[Mat198]

How long is the coax cable that comes with the camera?

How much is its bend radius?

[MartyG-RealSense]

Thanks very much @Mat198 for your questions about the D457. I have submitted them to my Intel RealSense colleagues to obtain the requested information.

[MartyG-RealSense]

Hi @Mat198 My Intel RealSense colleagues have provided the following D457 information.

1. The Fakra cable in the D457 box has a length of 1000 mm +/- 10

2. According to RG174 coax cable spec, the minimum radius is 9.9 mm.

4. Users can obtain a longer cable from the market. It is recommended to use “LEONI Dacar 302” for a 10 ~15 m long cable. An Intel RealSense colleague has tried RG174 cable on D457. The maximum length is 11 m, but Dacar 302 can reach 15 m.

[Mat198]

Thanks for your fast reply! Does the GMSL interface need a different implementation, or it's ready to plug and play in this Syslogic computer? https://www.syslogic.ai/jetson-agx-xavier/rugged-computer-rpc-rml-a3-e2

I saw ConnectTech products, but I'm aiming for an outdoor agriculture application that requires an IP rate of at least 65. https://connecttech.com/connect-tech-offers-built-in-support-on-the-newly-announced-intel-gmsl-realsense-depth-camera-d457/

[MartyG-RealSense]

A Maxim de-serializer is needed at host side to decode the streams. The Intel RealSense sales team can be contacted regarding the de-serializer component. A contact form for the sales team is at the link below.

https://www.intelrealsense.com/talk-to-sales/

I recommend selecting the Which Intel RealSense product option and then the Need advice sub-option.

For users who wish to perform tests with the D457, the Connect Tech product is recommended as no de-serializer is needed in a Connect Tech / D457 setup.

[Mat198]

Understood. Thanks!

[AndreV84]

do you supply interface / adapter to connect it to nvidia AGX devkit somehow?

[MartyG-RealSense]

Hi @AndreV84 If you have obtained a de-serializer to decode the D457's streams at host-side then information about how to obtain the reference driver package for AGX Xavier can be found at https://github.com/IntelRealSense/librealsense/issues/11010#issuecomment-1330218867

In regards to cabling, a 1000 mm / 1 meter FAKRA cable is supplied with the D457, as described at https://github.com/IntelRealSense/librealsense/issues/10984#issuecomment-1292320808

[AndreV84]

Hi @MartyG-RealSense Thank you for following up! However, the default Jetson AGX devkit doesn't have the GSML port for the cabling. Could you extend on how to connect the camera to the default nvidia Jetson AGX devkit hardware, please? Are there any particular cable+ adapter things or how else it could be connected?

[MartyG-RealSense]

My understanding is that the cabling connects to the de-serializer board, like the one at the link below that a RealSense user at https://github.com/IntelRealSense/librealsense/issues/11010#issuecomment-1306030505 chose for their D457.

https://www.leopardimaging.com/product/accessories/adapters-carrier-boards/li-gmsl2-ipx-deser/

In the data sheet document for the D457, it states "GMSL is based on SerDes technology, meaning using a serializer on the transmitter side and a de-serializer on the receiver side".

[AndreV84]

so for the camera connection we nee dto procure this board [ which is out of stock] https://www.leopardimaging.com/product/accessories/adapters-carrier-boards/li-gmsl2-ipx-deser/ ? are there any other boards we could buy for nvidia Jetson AGX devkit default implementation?

[MartyG-RealSense]

For the initial testing stage of a D457 project, Intel recommend using Connect Tech hardware so that purchasing a separate de-serializer is not required as de-serialization is built into the Connect Tech hardware.

https://connecttech.com/connect-tech-offers-built-in-support-on-the-newly-announced-intel-gmsl-realsense-depth-camera-d457/

If you would prefer to proceed directly to using a separate de-serializer component then it can be obtained through the Intel RealSense sales team, as described at https://github.com/IntelRealSense/librealsense/issues/10984#issuecomment-1292382772

Intel's de-serializer board can be plug onto the AGX Xavier board directly. The D457 connects to the FAKRA connector on this board through a GMSL cable.

I would not name further de-serializer products at present without confirmation that they are able to function with the D457. The RealSense user who purchased the LI-GMSL2-IPX-DESER product stated though that it contained MAX9296A technology, which should narrow down research for compatible de-serializer products.

For example, you could google for maxim deserializer "MAX9296A" (including the quotation marks so that relevant products are included in the search results).

The linked-to product listing itself confirms this particular Maxim component.

[AndreV84]

@MartyG-RealSense Do we order a de serializer from intel realsense somehow directly through sales easily? or we should look to order elsewhere e.g. from Leopard imaging? The latter [LI] listed these:

We have below adapter boards and cable which can connect GMSL2 cameras to Jetson AGX Xavier, but I am sure if this deserializer adapter (with MAX9296) is compatible with Intel d457 camera.

LI-GMSL2-IPX-DESER
FAW-1233-03 cable
BLI-JXAV-MIPI-ADPT-4CAM```

But if Intel realsense can supply sale of de serializer it maybe even better option?

Also are there adapters for Jetson NX devkits by nvidia?

[MartyG-RealSense]

Contacting Intel sales regarding the Intel de-serializer product that can plug directly onto AGX Xavier would be the safest option, especially in regard to your question of how to interface the de-serializer with the AGX board.

Intel's guidance is to use a Maxim de-serializer rather than a specific model of Maxim de-serializer, indicating that if the de-serializer supports GMSL then it should be able to work.

[AndreV84]

@MartyG-RealSense Thank you for following up I think I have filled the contact with sales form Maybe they will follow up sometime in the future to reach back to me regarding the procurement of the de serializer. Does it take many days to get the contact from sales?

[MartyG-RealSense]

Bearing in mind that it is December 23 at the time of writing this, if you have not had a response in a couple of weeks then let me know and I will follow it up.

[AndreV84]

@MartyG-RealSense Will these CSI-> GSML adapters provided by Intel or LI work with Nvidia Jetson Orin devkit by Nvidia? There is no such support for NX devits, right? But there is support for AGX. How about Orin Devkits?

[MartyG-RealSense]

The D457's Jetson reference driver support is designed to work with a de-serializer on AGX Xavier at this time.

[jast1982]

Hi!

Is there any documentation for bringup? I have a Rudi-AGX and D457 cameras on my desk but realsense-viewer (v2.53.1) doesn't find them with their Jetpack & BSP 4.6.1 installed (4.6.1 was recommended by ConnectTech). I installed the base-image, do I maybe need to select any of their AR** or IMX 390 images? A USB3 D450 works fine btw with USB 3.2 connection.

Looking forward to any hints how to get them up and running...

Thanks!

[MartyG-RealSense]

Hi @jast1982 I will discuss your issue with my Intel RealSense colleagues. Thanks very much for your patience!

[MartyG-RealSense]

Hi again @jast1982 My colleagues advised that first, you need to use the latest RealSense Viewer version. I believe that you are already doing this as you are using SDK version 2.531.

Secondly, you need to obtain the D457 driver from Connect Tech. Intel unfortunately cannot help with this and you need to contact Connect Tech directly.

[AndreV84]

@MartyG-RealSense Could you confirm if we ordering these three units for jetson AGX devkit will be able to use it for d457 connection? Thanks

LI-GMSL2-IPX-DESER x1

FAW-1233-03 cable x1

LI-JXAV-MIPI-ADPT-4CAM x1

[AndreV84]

@MartyG-RealSense is it the deser board from intel? 842, no developers discount? :) sales yet remains silent https://store.intelrealsense.com/buy-intel-realsense-des457.html

[MartyG-RealSense]

The de-serializer board on the store link is the Intel one, though I would recommend contacting the Intel RealSense sales team to confirm pricing. As mentioned above at https://github.com/IntelRealSense/librealsense/issues/10984#issuecomment-1364041699 - please allow another week for a response to your submitted query and I will follow it up for you if you have not received a response after that. Thanks very much for your patience!

In regard to the components list, it was confirmed by my Intel RealSense support colleagues at https://github.com/IntelRealSense/librealsense/issues/11010#issuecomment-1306169640 that LI-GMSL2-IPX-DESER could be used with D457.

in regard to a cable, my colleagues recommended at https://github.com/IntelRealSense/librealsense/issues/10984#issuecomment-1292320808 using a LEONI Dacar 302 cable or a cable that is compliant with RG174.

The D457 camera should connect onto the de-serializer board. If using Intel's de-serializer board then it should attach directly onto the AGX Xavier board, as described at https://github.com/IntelRealSense/librealsense/issues/10984#issuecomment-1363709490

As you quoted the FAW-1233-03 cable though, it looks as though you are referring to a different kind of cable to the one that connects the D457 to the de-serializer.

Can you provide information please about the purpose that the LI-JXAV-MIPI-ADPT-4CAM and LI-JXAV-MIPI-ADPT-4CAM would have in your project, please? Thanks!

[jast1982]

Hi again @jast1982 My colleagues advised that first, you need to use the latest RealSense Viewer version. I believe that you are already doing this as you are using SDK version 2.531.

Secondly, you need to obtain the D457 driver from Connect Tech. Intel unfortunately cannot help with this and you need to contact Connect Tech directly.

Hi!

So, I was able to progress somewhat: Downloading the Realsense specific BSP from here https://connecttech.com/supported-cameras/ installing it according to the normal instructions and afterwards adding this line to /boot/extlinux/extlinux.conf:

FDT /boot/tegra194-agx-cti-AGX104-INTEL.dtb

enabled at least two out of my 4 connected realsense cameras on my system. I can see /dev/video0 to 11 and /dev/video2 and /dev/video8 play a video stream (in wrong colors) in VLC. Unfortunately the realsense-viewer outputs this:

29/12 10:34:54,968 INFO [548026171408] (context.cpp:382) Found 2 RealSense devices (mask 0xff) 29/12 10:34:55,251 INFO [548026171408] (rs.cpp:2701) Framebuffer size changed to 2688 x 756 29/12 10:34:55,252 INFO [548026171408] (rs.cpp:2701) Scale Factor is now 1 29/12 10:34:55,350 ERROR [547922886432] (librealsense-exception.h:52) xioctl(VIDIOC_S_EXT_CTRLS) failed Last Error: Invalid argument 29/12 10:34:55,356 WARNING [547922886432] (rs.cpp:310) null pointer passed for argument "device" 29/12 10:34:55,356 WARNING [547922886432] (rs.cpp:2704) Couldn't refresh devices - xioctl(VIDIOC_S_EXT_CTRLS) failed Last Error: Invalid argument (V2.53.1, tried as user and with sudo)

Any ideas?

[MartyG-RealSense]

Hi @jast1982 As it is a xioctl error, it would indicate to me that there is a kernel conflict. You could try building the SDK from source code with CMake with the RSUSB backend installation method to confirm whether or not this is the cause, as RSUSB bypasses the kernel and so is not dependent on Linux versions or kernel versions and does not require kernel patching. RSUSB is not ideally suited to multiple cameras however, as a kernel-patched build works better with multicam.

[jast1982]

@MartyG-RealSense That was my hope as well. Already compiling...

[AndreV84]

@MartyG-RealSense It appears that the design presumes that LI deser is connected to the 4x cameras adapter It doesn't appear that it can be connected directly to Jetson AGX, can it? http://www.leopardimaging.com/uploads/LI-IMX390-GMSL2-XAVIER-xxxH_datasheet.pdf

[AndreV84]

@MartyG-RealSense from LI regarding the purpose

The LI-JXAV-MIPI-ADPT-4CAM is a MIPI adapter which can connect GMSL2 camera and LI-GMSL2-IPX-DESER to the Jetson AGX.
Please make sure your d457 camera includes a GMSL2 interface (MAX9295). You may also need to build the Jetson driver for this camera kit.

[MartyG-RealSense]

@AndreV84 If you have an Intel NDA agreement then I recommend submitting a Zendesk support ticket to Intel for these technical questions about sourcing and attaching together components for use with D457. Alternatively, if you intend to use components that are all made by LI then LI are likely to be able to assist, like with their response above.

[AndreV84]

@MartyG-RealSense As far as I understand LI can not confirm that their solution works with d457 specifically as they did not do such testing specifically. But they say it compatible with the chip. Could you extend on how to get through NDA step to Zendesk ticket creation, please? community.intel.com/t5/Items-with-no-label/Multiple-Camera-Positions-Calibration/m-p/1443270#M17774

[AndreV84]

@MartyG-RealSense Could you confirm that " Intel D457 camera includes GMSL2 serializer (MAX9295)." ? as LI points out "If the D457 camera includes a GMSL2 serializer, this camera should be compatible with our MAX9296 deserializer adapter (for hardware). However, you still need to build a driver for Jetson AGX Devkit and D457 camera. " My concern is who supplies the driver ? intel or LI for that design? Thanks

[MartyG-RealSense]

@AndreV84 I will make enquires to my RealSense colleagues about your question. Thanks very much for your patience.

[AndreV84]

@MartyG-RealSense I signed for Zendesk account could you extend if Suite or Support type of subscription is required to create ticket to intel from zendesk account? Thanks

[MartyG-RealSense]

I have referred your latest enquiry to my colleagues.

[AndreV84]

@MartyG-RealSense Thank you for following up I did not hear back from realsense sales pertaining the intel de serializer ordering procedure or regarding price confirmation; is the listed at backorder page at librealsense price valid for the de serializer? Thanks

[MartyG-RealSense]

@AndreV84 Please send me an email so that I have your email contact to pass to my Intel RealSense colleagues.

[AndreV84]

@MartyG-RealSense on Jetson AGX devkit with intel dse serializer, whoch Jetson OS supported? the latest Jetpack 5.x? or only 4.x? Thanks

[MartyG-RealSense]

JetPack 5.0.2 support has only just been introduced into the RealSense SDK in its development branch and will therefore be included in the next SDK release (likely numbered 2.54.0). At the time of writing this, 2.53.1 supports JetPack 4.6, though RealSense users have already been able to unofficially use JetPack 5 successfully with the SDK (though with IMU problems but with depth and color working fine).

https://github.com/IntelRealSense/librealsense/tree/development

[AndreV84]

nvidia already released Jetpack 5.1 @MartyG-RealSense isn't it supported in the development branch of sdk? Thanks

[MartyG-RealSense]

SDK support of JetPack is not synchronized with new JetPack releases and official support for specific JetPack versions is added over time (though as mentioned above, currently unsupported JetPack versions can sometimes work).

[jrecasens]

Looks like it is very challenging to work with the GMSL/FAKRA interface.

Someone in this thread owns and have used the RealSense D457? would you please confirm that the USB port cannot be used for development purposes (or like the D455)?

[jast1982]

Unfortunately my setup (ConnectTech Rudi + 1-6 Realsenses) still doesn't work due to an init issue with the cameras. Also ConnectTech doesn't seem to have a clue. Does anyone outside of Intel has a working setup?

[AndreV84]

@jrecasens what are the error messages? can you attach dmesg log outputs? is it sdk errors or camera driver errors? in the former case did you try building the sdk from sources from development brannch? or installed from apt repository?

[jast1982]

I have this issue https://github.com/IntelRealSense/librealsense/issues/10984#issuecomment-1367191641 and no idea how to fix it. I tried with master and mipi branch with the same results. Cameras are detected, but open in librealsense fails.

[MartyG-RealSense]

The Leopard Imaging de-serializer board that my Intel RealSense colleagues confirmed is able to be used with D457 is now listed as In Stock at the time of writing this.

https://www.leopardimaging.com/product/accessories/adapters-carrier-boards/li-gmsl2-ipx-deser/

[AndreV84]

@MartyG-RealSense did your colleague used it directly with AGX devkit somehow or it necessarily required an extra 1 to 4 cam MIPI adapter by LI in order to run?

[MartyG-RealSense]

I assume that it was a general statement of compatibility based on the store description rather than it having been actually purchased and tested on specific hardware.

[AndreV84]

The Leopard Imaging de-serializer board that my Intel RealSense colleagues confirmed is able to be used with D457 is now listed as In Stock at the time of writing this.

---

according to the manufacturer [LI] it [li-gmsl2-ipx-dese] won't work [ won't conenct] to AGX Jetosn devkit without an extra AGX CSI MIPI 4 cameras adapter sold separately that won't be in the stock mid March, as sales of LI pointed out

Moreover, there are some rumors from third party vendors that despite the statement by the manufacturer that the aadapter won't work with AGX Jetson without an additional adapter , that li-gmsl2-ipx-deser can be wired to aGX devkit with just an external 12V power + 2 CSI wires to wire the li-gmsl2-ipx-deser to Jetson AGX directly. But vendor/ manufacturer doesn't list such design as supported without separate 4x cameras adapter

[MartyG-RealSense]

@AndreV84 Thanks for the information about the adapter.

Advice from my Intel RealSense colleagues at https://github.com/IntelRealSense/librealsense/issues/11010#issuecomment-1301807128 stated that the de-serializer MAX9296A (the Leopard Imaging product uses MAX9296) can be paired with the MAX9295A serializer built inside the D457. As the Leopard Imaging board contains the appropriate de-serializer component, this is likely the basis of confidence that it can work when paired with D457.