marek-simonik / record3d

Accompanying library for the Record3D iOS app (https://record3d.app/). Allows you to receive RGBD stream from iOS devices with TrueDepth camera(s).
https://record3d.app/
GNU Lesser General Public License v2.1
379 stars 55 forks source link

How to access conf values when streaming? #81

Closed alik-git closed 3 months ago

alik-git commented 4 months ago

Hi there, thanks for this awesome repo!

Is it possible to access the conf values for depth when streaming? or is that only possible when saving the r3d file?

Please let me know, thanks!

marek-simonik commented 4 months ago

Hi,

the depth confidence files are currently available only for the recorded sequences (in .r3d files), but adding the confidence maps also to the USB stream is on my TODO list for a future version of Record3D.

marek-simonik commented 3 months ago

This feature has been implemented in Record3D 1.10, which necessitated breaking changes to be done to the record3d library. Please update to Record3D 1.10 (or newer) and also update to record3d library version 1.4.

You can access confidence maps as shown here: https://github.com/marek-simonik/record3d/blob/0c6e7692da100aad2d5195e307f01e75427a6a2a/demo-main.py#L52

A confidence map is a uint8 image, where each pixel can have one of the following values representing the confidence of estimated depth value in the corresponding depth map:

Note that confidence maps are available only for LiDAR streams (otherwise the confidence map has resolution 0x0 px).