NaN values in depth data

[scotgopal]

I've realised that the 32FC1 encoded depth data has some NaN values. May I know what is the significance of this data? And what is the situation that may cause Zivid to record this data?

Hole filling was enabled prior to recording this data, so that is not helping to avoid NaN values in depth data. Is there any other config that I can set to avoid it from appearing in my depth data?

Thank you.

[scotgopal]

Hi there. Can anyone of you help with this issue please?

@ajlandau @nedrebo @eskaur

[eskaur]

Hi @scotgopal !

A Zivid point-cloud is a "Structured Point Cloud" with a set of (X,Y,Z) values for each pixel in the camera. There are several reasons why a pixel may have (X,Y,Z) = (nan,nan,nan). The most common ones are:

• The point has been filtered out by one of the filters (outlier fillter, noise filter, reflection filter, etc).
• The point in your scene that the given pixel is looking at is in the "projector shadow", i.e. the projected pattern is blocked from reaching that part of the scene.

While you could in theory get rid of the first cause by turning off all filters (not a good idea generally), the second cause will always be present: It is impossible for us to successfully reconstruct (X,Y,Z) for all pixels in the camera for any given scene due to the "projector shadow" effect.

Thus, there will always be some NaN Z-values in a Zivid point cloud, and the user has to accommodate for that in whatever algorithm consumes the point cloud.

Tip: If your algorithm is fine with an Unstructured Point Cloud (basically just a linear list of points) you can get a NaN-free point cloud by looping over all our points and only copying over the ones that are non-NaN.