Best RealSense for fast moving object

[snopytas]

Hi once again,

currently we own a D435 but we are planning to buy another RealSense. The use case is to track a thrown ball. With the D435 we faced the following problems:

1) Rolling shutter in the RGB Stream, so that the detection of the ball in the color frame is a bit inaccurate.

2) The Depth is inaccurate for distances greater than ~2.5m.

3) When the ball is moving fast, in the color frame the ball is not at the same position as on the aligned depth frame. Therefore if the ball has coordinates u,v in the color frame and we want to know its depth it is necessary to "scan" the region around u,v (for example u+-15, v+-15) to get a suitable depth value. The situation is better with 90FPS but not fully resolved.

I've read that the D455 has global shutter (solves 1), a greater depth range (solves 2) and the infrared sensors can produce both colored images and the depth image, such that they are perfectly aligned (solves 3). So I guess that the D455 could be perfect to track thrown balls. Is this correct?

EDIT: After reading I think the third point is not true: with the D455 I can't get perfectly aligned frames, it is the D415 which has this "feature".

Another option would be to buy the L515 but I don't really understand what's the advantage over the D455. Maybe you could give me some hints.

Thanks.

[MartyG-RealSense]

Hi @snopytas Yes, the D455 camera model supports greater range, has fast Global shutters on both the RGB and depth sensors, has 2x the accuracy over distance of the D435 / D435i (the same accuracy at 6 m as D435 / D435i has at 3 m) and has improved depth-color alignment due to the RGB and depth sensors being placed on the same internal stiffener.

D455 can also output an RGB8 color image from its left infrared sensor if the camera has firmware 5.12.8.200 or newer installed, though it may not be the same in appearance as a color image from the RGB sensor.

https://github.com/IntelRealSense/librealsense/issues/7897#issuecomment-736370516 https://github.com/IntelRealSense/librealsense/issues/7870#issuecomment-735430816

A factor to consider when selecting the D455 is that it has a minimum depth sensing distance of 0.4 m compared to 0.1 m on D435 / D435i, so the D455 cannot be positioned as near to objects and surfaces. This should not be a problem if you are scanning a ball that is more than 0.4 m from the camera though.

The D455 should certainly provide improved depth-color alignment. I would not give a guarantee of perfection for any camera model though, simply because of the range of environmental and programming factors that can affect the image that is output.

In regard to the L515, it is suited for indoor use in controlled lighting conditions, preferably without windows to let sunlight through (as it is sensitive to infrared light sources). You may also experience accuracy drift with it beyond 4 m distance. The D455 will be an appropriate choice if you require long range, strong support for depth-color alignment and fast capture for both depth and color.

[snopytas]

Thank you! Does the L515 have any advantages over the D455, except for the longer depth range?

[MartyG-RealSense]

L515 can generate 23 millon accurate depth points per second and it has motion blur artefact reduction, though its depth and infrared FPS speed is fixed at 30 FPS compared to the wider range of FPS speeds supported on 400 Series cameras.

Intel have published a short information video about the L515's features that is available at the YouTube link below.

https://www.youtube.com/watch?v=Rx-75ZEAMeg

[snopytas]

I think we will buy the D455. Except for the higher range I don't see any benefit with the L515. Thank you very much @MartyG-RealSense

[MartyG-RealSense]

Thanks very much @snopytas for the update. I'd just like to clarify as a final point that the D455 has a greater depth measuring range than the L515.

[snopytas]

@MartyG-RealSense Hi, I want to add that we obtained a D455. The bigger range is great. But whats really really cool is that the infrared frame is colored. The quality of the infrared frame is somewhat lower than of the RGB sensor, but it is sufficient for our applications. In the end can I get a color frame which is perfectly aligned to the depth frame. That's my favorite feature of the D455.

[MartyG-RealSense]

Glad to hear that you are enjoying the benefits of the D455 :) Yes, frames from the left infrared sensor are perfectly aligned with the depth map, as described in the section of Intel's camera tuning guide linked to below.

https://dev.intelrealsense.com/docs/tuning-depth-cameras-for-best-performance#section-use-the-left-color-camera

This is true for all the 400 Series camera models, though only the D415 and D455 can output color from the left infrared sensor.

[snopytas]

Hi, the specification of the D455 tells me that the depth error is <2% within 4 meters. Could you maybe tell me under which conditions or presets this holds? At least I have the impression that this specification isn't always true.

[MartyG-RealSense]

Accuracy can be influenced by a range of factors such as lighting, environment and the materials and colors of the object / surface that the camera is observing.

If you know what to look for, an RGB image of the scene can provide indicators at a glance of factors in the scene that may be causing accuracy problems. This could include dark grey / black surfaces, fluorescent lights such as ceiling strip lights, and repetitive horizontal and vertical patterns (such as ceiling / floor tiles, a row of fence posts, a vertical stack of slats in a window blind, etc). https://github.com/IntelRealSense/librealsense/issues/6713 discusses possible methods to minimize the effect of repetitive patterns,

You could also try using the Medium Density preset which provides a good balance between accuracy and the amount of depth detail ('fill rate') in the scene. The High Accuracy preset may remove too much depth detail from the scene.