search

czq142857 / IM-NET-pytorch

PyTorch 1.2 implementation of IM-NET.
Other
196 stars 27 forks source link

Reproducing for Point Cloud to Mesh Reconstruction #7

Closed Angps1995 closed 3 years ago

Angps1995 commented 3 years ago

Hi,

I am looking to convert a point cloud to mesh using IMNet.

I am able to reproduce the paper on shapenet dataset. However, when trying to train from scratch or applying transfer learning on my own dataset, I am unable to reproduce the good reconstruction result.

Steps taken so far:

  1. Convert Point Cloud to Voxel Grid
  2. Use the point sampling script to sample the point-value pairs and voxels
  3. Train the model

My hypothesis is that my point cloud to voxel conversion is not as good and smooth as the original voxels you have from shapenet. Would you happen to have any idea on how to properly voxelise and get the point-value sampling from a point cloud?

czq142857 commented 3 years ago

Hi,

The current training loss does not support training from point clouds; it requires ground truth voxels or point-value pairs for training.

Training implicit field models from point clouds itself is an ongoing research area. I would recommend you to look at this work for ideas: SAL: Sign Agnostic Learning of Shapes from Raw Data https://arxiv.org/abs/1911.10414

However, if your point clouds are dense enough, you can always obtain their correct voxel models for training by:

  1. Convert point clouds to voxel grids with appropriate resolution (say 64, the smaller the better for floodfill).
  2. Run a floodfill algorithm to make sure the voxels inside a shape are '1's, and the voxels outside are '0's.
  3. Use the point sampling script to sample the point-value pairs and voxels
  4. Train the model
Angps1995 commented 3 years ago

I see. Thank you for your reply, it is very helpful!! :)