search

tangjiapeng / SA-ConvONet

ICCV2021 Oral SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks
MIT License
89 stars 5 forks source link

Is there any method to preserve the original color of the point cloud on the output model? #5

Open TKPTKPTKP opened 2 years ago

TKPTKPTKP commented 2 years ago

Hi, @tangjiapeng I'm working on a study project to reconstruct the outdoor environment via point cloud data and trying to use your impressive method on it. I just found out that the model output .off file, which is not supported to save the color. Since I want to make an end-to-end pipeline, I'm wondering if there is a way to preserve the original color of the point cloud on the output model?

tangjiapeng commented 2 years ago

Hi, if you want to reconstruct colored surface meshes, you can extend our model to learn color fields that are defined at the near surface region. Give a point sampled in near surface region, you query the corresponding feature vectors from volumetric grids. Based on the feature vectors, you predict the color values of this point. To train such a model of color prediction, you can prepare the point-color pairs in this way: you can introduce an offset N (0, d) to the surface points along the surface normal N so that the color can be defined not only on the exact surface but also on the 3D space around it. After pretraining such a color prediction module, you can also run the test-time optimization to further improve the generality to given colored point clouds.