search

TRI-ML / packnet-sfm

TRI-ML Monocular Depth Estimation Repository
https://tri-ml.github.io/packnet-sfm/
MIT License
1.23k stars 242 forks source link

3D point cloud reconstruction #195

Closed chapa17 closed 2 years ago

chapa17 commented 2 years ago

Hello everyone,

I want to construct 3D point cloud for which I require 3D points. I am using def reconstruct() from camera.py file but I am getting a point cloud which has some different scale. The depth map looks quit good but the point cloud is really bad. I am not able to figure out whether it is the problem with my implementation or it is the same.

If anyone has tried reconstruction I would be happy to learn.

Thank you!

VitorGuizilini-TRI commented 2 years ago

If you are using a purely self-supervised model, there is no metric scale in the depth map. Is that what you mean when you say different scale, and that the pointcloud is really bad?

chapa17 commented 2 years ago

I guess I have figured out the cause. It is the cloud pixels in the 3D cloud that are stretched out and not in proper position. Which seems to be a drawback of self-supervised methods anyways.

Thank you for your help!

chapa17 commented 2 years ago

I am sorry to reopen the issue again. The point cloud I have generated using reconstruct() method does not lie exactly on the ground plane. By ground plane I mean that when I visualize the point cloud in Blender, it does not lie above the grid. My understanding is that the point cloud lies on the ground plane grid. I have attached the snapshot of my point cloud and an example point cloud. The values which are generated are not aligned. Generally X-axis faces front direction, Y-axis faces left direction and Z-axis faces top direction. But in my case all are in different direction

Point cloud generated from packnet-sfm

Screenshot from 2021-12-15 09-20-40

Example point cloud

Screenshot from 2021-12-15 09-28-30

Can you please let me know where I went wrong?