search

marek-simonik / record3d

Accompanying library for the Record3D iOS app (https://record3d.app/). Allows you to receive RGBD stream from iOS devices with TrueDepth camera(s).
https://record3d.app/
GNU Lesser General Public License v2.1
379 stars 55 forks source link

Are the world coordinates in the camera poses outputted by record3d consistent with the scale of the depthmap? #43

Closed andrewsonga closed 1 year ago

andrewsonga commented 1 year ago

As I asked in the title, are the world coordinates in the cam-to-world transformation from record3d scale-consistent with the depthmap outputted by record3d?

Thank you. Andrew.

marek-simonik commented 1 year ago

Hello Andrew, yes, they should be scale-consistent. The depth maps hold depth values expressed in meters and the LiDAR world position reported by Apple's API (which is what Record3D saves) is expressed in meters too.

I haven't checked this in the latest Beta and non-Beta iOS versions, but based on my past experience, both the depth map data and the reported ARKit positions should be in real-world scale (expressed in meters), so there should be no scale difference.

However, please note that the LiDAR depth maps that Apple provides are generated by Apple's neural network that fuses raw LiDAR data with color camera data, so the depth measurements are not particularly precise. Try covering all camera lenses by your finger(s) so that only the LiDAR sensor is not obstructed and you will see how much does the depth data change.

Did you notice any scale inconsistency?

marek-simonik commented 1 year ago

This issue has been inactive for several months, so I'm closing it. Feel free to reopen it if you have follow-up questions.