search

isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
10.75k stars 2.23k forks source link

Real time surface reconstruction: point cloud --> matching cubes -->mesh? #4442

Closed 1939938853 closed 2 years ago

1939938853 commented 2 years ago

Hello,

The current surface reconstruction methods (Alpha shapes, ball pivoting and Poisson surface) are quite slow. Just wonder if it is possible to take the approach like the one below and finally using matching cubes?

point cloud --> RGBD --> TSDF volume --> triangular mesh

I see there is a function called ProjectToRGBDImage() in PointCloud class. So the approach seems possible. Can anyone give me a sample code? How to get or set the camera PinholeCameraIntrinsic parameters?

Thank you!

yxlao commented 2 years ago

We have the "RGBD --> TSDF volume --> triangular mesh" pipeline already.

For "point cloud --> RGBD", we'll need to determine the occlusion relationship given the camera parameters. Also, the point cloud may need to be dense enough to get a good quality RGBD projection.

1939938853 commented 2 years ago

Hi,

My point cloud can be scaled in millimeter. I want to project it to x-y-plane, so it can be, for example, a 800x1000 depth image. On the z-axis, I can take only the max z-value if on the same grid there are multiple ones. In this case, how to determine PinholeCameraIntrinsic parameters?

I think that the main obstacle for me to use the surface reconstruction pipeline of "point cloud --> RGBD --> TSDF volume --> triangular mesh" is that I don't know how to choose PinholeCameraIntrinsic parameters. Here I need some examples.

About cloud point density, I think, maybe it is possible to do some kind of "interpolation" morph operation on the depth image?

Thanks

1939938853 commented 2 years ago

PCL has match cubes surface reconstruction function. For Open3D, it is just one step (not so much work) left. Hopefully some developer can have the interest to implement it.

Thanks

theNded commented 2 years ago

For point cloud to RGBD, pinhole camera intrinsic is decided by which sensor you want to simulate. You can start with the example I put in #4464. The overall idea looks reasonable but the performance could be significantly affected by the actual data.

We have MarchingCubes from TSDF voxel grids. I don't regard it as "just one step (not so much work) left" to migrate it to pure point clouds with RBF kernels. Hopefully you can contribute to implementing it if you have the interest.