search

TRI-ML / packnet-sfm

TRI-ML Monocular Depth Estimation Repository
https://tri-ml.github.io/packnet-sfm/
MIT License
1.21k stars 241 forks source link

Metric scale for monocular image #206

Open chapa17 opened 2 years ago

chapa17 commented 2 years ago

Hello,

I am trying to run the code on my own dataset. I am using a monocular camera. The depth map when reconstructed to 3D point cloud has ground at some elevation which is different for every point. I am aware that I need to scale the point cloud. But I am not sure how to obtain the metric scale for depth.

It would be really great if someone can provide some hint to this.

Thank you!

VitorGuizilini-TRI commented 2 years ago

The monocular pointclouds should be accurate up to a single scale factor, so it is strange that each ground point has a different elevation. They should be consistent, just not metrically accurate. To obtain metric scale you can either use GT depth for supervision or our velocity weak supervision in addition to self-supervision. I hope that helps!

chapa17 commented 2 years ago

Yes, I am trying to use the pre-trained model with velocity supervision i.e. PackNet, Self-Supervised Scale-Aware, 192x640, CS → K. I hope it is the same you are trying to point at.

Also I get the depth values as high as 85. Can you please confirm the unit of this values (mm or cm)?