search

marek-simonik / record3d

Accompanying library for the Record3D iOS app (https://record3d.app/). Allows you to receive RGBD stream from iOS devices with TrueDepth camera(s).
https://record3d.app/
GNU Lesser General Public License v2.1
382 stars 55 forks source link

Several questions on updated resolution #30

Closed zehuiz2 closed 2 years ago

zehuiz2 commented 2 years ago

Hi, I wonder if you've updated both LiDAR & FaceID depth resolution?

depth_img = depth_img.reshape((1280, 960)) # For a FaceID camera 3D Video depth_img = depth_img.reshape((512, 384)) # For a LiDAR 3D Video

Is the above correct? Another three questions:

  1. Is it possible you could update the FaceID RGB resolution? It is still 640*480.
  2. It seems the LiDAR confidence resolution is not updated?
  3. What are the units of the depth measurements? I assume it is mm?

Thank you very much!

Originally posted by @zehuiz2 in https://github.com/marek-simonik/record3d/issues/7#issuecomment-1024665589

marek-simonik commented 2 years ago

Hi, I have not updated resolution of any output — all resolutions stay the same (as far as I know, Apple has not increased depth resolutions in iOS 15, so 192x256 px for LiDAR and 480x640 px for FaceID are the maximums, therefore I do not plan any changes yet). The resolution of LiDAR confidence maps should also be 192x256 px. To answer your third question: the depth measurements are in meters.

zehuiz2 commented 2 years ago

Thank you for the reply. I figured it out: I mistakenly used: depth_img = np.frombuffer(decompressed_bytes, dtype=np.uint8)

marek-simonik commented 2 years ago

Thank you for the reply. I figured it out: I mistakenly used: depth_img = np.frombuffer(decompressed_bytes, dtype=np.uint8)

That is not the correct way of loading the depth image. Please see this sample script: https://github.com/marek-simonik/record3d/issues/7#issuecomment-736573410

The issue with the snipped you posted is that you interpret the bytes as uint8. You should instead load depth maps as float32 — then you will get depth values in meters.

Only the confidence maps are to be loaded as uint8.