search

TRI-ML / packnet-sfm

TRI-ML Monocular Depth Estimation Repository
https://tri-ml.github.io/packnet-sfm/
MIT License
1.24k stars 243 forks source link

Question about camera poses #77

Closed janhuenermann closed 4 years ago

janhuenermann commented 4 years ago

Hey,

thanks for putting this repo up. It's a very good resource. I am currently pursuing research on monocular depth prediction and have a question about your use of camera poses inside of the geometry functions.

In the function Camera.reconstruct that reconstructs a point cloud from a depth map, you are using the pose Twc. I am confused, as this pose maps from world to camera frame but the points Xc you are transforming are in camera (and not world) frame. To me this raises the question why you are not using the pose Tcw (camera to world) here, as the reconstructed point cloud is supposed to be in world frame also.

Similarly, the pose Tcw is used in Camera.project even though the input point cloud is specified to be in world frame. Why do you not use Twc here?

I suppose, as these matrices are swappable ($Tcw = Twc^{-1}$) and the underlying value is learned by the network, it does not make a difference. Nonetheless, I was wondering if you had any intentions with this convention?

VitorGuizilini-TRI commented 4 years ago

I think we agree, but look at the notation in a slightly different way: Tcw maps from world to camera. Because it multiplies on the left, we multiply Twc * Xc to get Xw, does that make sense? Let me know if there is anything still wrong if you follow this convention!

janhuenermann commented 4 years ago

Makes sense. Thanks for taking the time!

surfii3z commented 4 years ago

I think we agree, but look at the notation in a slightly different way: Tcw maps from world to camera. Because it multiplies on the left, we multiply Twc * Xc to get Xw, does that make sense? Let me know if there is anything still wrong if you follow this convention!

Hi @VitorGuizilini-TRI, then I noticed that the comment is probably misleading? Tcw : Pose Camera -> World pose transformation

VitorGuizilini-TRI commented 4 years ago

@surfii3z You are right, I will fix that, thank you!