Depth map generation service

[banafshebamdad]

Hello, First of all, thank you for the incredible tools and support! I am currently working on a project that involves generating depth maps using stereo images from Meta Aria glasses. While the tools provided are extremely useful, I noticed that there isn't a service for generating depth maps directly. I wanted to inquire if there are any plans to add a service for depth map generation in MPS in the near future. Thank you for your consideration, Banafshe Bamdad

[SeaOtocinclus]

Hello @banafshebamdad, Thank you for your message and interest. Today the only way to retrieve depth information with MPS is to use the SLAM service that provides semidense points information. The observations file allows you to retrieve in which frame a point is visible and so build sparse depth maps. See https://facebookresearch.github.io/projectaria_tools/docs/data_formats/mps/slam/mps_pointcloud for more information.

BTW, have you try any method yet on computing depth maps from the stereo images?

• using temporal data could provide better results, since the baseline between the SLAM images is short.

[banafshebamdad]

Thank you for your informative response. The suggestion to leverage the SLAM service with observations file is valuable. In my project, depth information will serve as ground truth for an RGB-D SLAM system, therefore high-accuracy depth information is crucial. That is why I reached out to explore potential solutions in MPS. Based on your post, I explored Multi-View Stereo vision (MSV) technique and deep learning-based depth estimation approaches. While I attempted using MVS, I came across your explanation regarding the Aria camera orientation being optimized for field of view rather than stereo depth recovery. This likely explains the minimal overlap I'm observing between captured stereo image pairs.

Thank you again for your time and support. Banafshe

[SeaOtocinclus]

@banafshebamdad Closing the issue, feel free to re-open if you have more question in this area.