hustvl / 4DGaussians

[CVPR 2024] 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
https://guanjunwu.github.io/4dgs/
Other
1.88k stars 147 forks source link

How to output the dense point cloud or high quality mesh #26

Open junhua-l opened 8 months ago

junhua-l commented 8 months ago

Hi, I want to export the dense point cloud or high-quality mesh.

I tried to export mesh, but the accuracy is very poor.

Point cloud I started directly from the output point cloud, and then SH2RGBto the normal point cloud. However, I found that the point cloud derived in this way is far inferior to traditional 3D reconstruction methods, such as dense reconstruction colmap. (1) I increased the number of Gaussian during training by changing densify_from_iter, but the resulting point cloud quality was still not good (in the case that the number of point clouds was much larger than the point clouds generated by dense reconstruction), I think simply increasing the number of Gaussian is not a good method. (2) I replaced the initial random points3D.ply with the point cloud generated by my dense reconstruction. I found that because of the nature of Gaussian, the number of meaningful points has been decreasing, and then a lot of noise points are added, so the point cloud obtained after training is still not as dense reconstruction.

I would like to know if there is any idea to export a high-quality point cloud or mesh. This may not be the focus of this paper, but it could also be important.

Here is a Chinese translation of my issue:

Hi 你好,我这里想导出dense的point cloud或者high-quality的mesh

我尝试导出了mesh,但精度非常差。

点云我一开始是直接从output的点云,再SH2RGB转成正常点云。但我发现这样导出的点云远不如传统三维重建的方法,比如dense colmap中的reconstruction。(1) 我通过改变densify_from_iter来增加训练时高斯的数量,但最后导出的点云质量依然不如 (在点云数量远大于dense reconstruction产生的点云的情况下), 我觉得单纯增加高斯的数量不是什么好方法。 (2) 我把最开始随机产生的points3D.ply 改成了我dense reconstruction产生的点云。我发现因为高斯的性质,点的数量一直在减少,然后多了很多noise,这样训练之后得到的点云依然不如dense reconstruction. 我想知道有没有什么思路可以导出high-quality的点云 or mesh

guanjunwu commented 8 months ago

Now 4D-GS do not focus on dense point clouds generation (maybe we will do it later) Anyone knows?

junhua-l commented 8 months ago

Thank you! I found that there were too many noise point clouds in the periphery, and the density of the central point cloud was not high enough. I tried post-processing, and the effect was mediocre. I want to know how to adjust hyperparameters to solve this problem. Any possible advice or practice would be very helpful to me :)

Also an interesting point is that a dense initialization point cloud does not seem to improve the quality of the final Gaussian, I am not sure if it is because my implementation is wrong. Has anyone else found this?