heheyas
commented
11 months ago Hi ingra14m,
Thank you for your interest. In fact, we encountered similar issues during the early experiments, especially when dealing with a large number of points. Point-E consistently generated relatively simple shapes, which resulted in models that lacked fine details. Therefore, we proposed a two-stage optimization approach. In the geometry optimization phase, we incorporated Point-E's SDS loss into the training process. We didn't expect Point-E to directly provide highly precise geometric information but rather considered it as a supplementary tool for 2D SDS. It comes into play when noticeable collapsing occurs, helping to rectify such deviations. In the appearance refinement phase, we assumed that the rough shape obtained in the previous steps possessed certain geometric priors. This allowed us to obtain better models than without Point-E optimization. However, it's worth noting that this approach is still far away from eliminating geometrical denigration, as mentioned in the failure cases outlined in our paper.
ingra14m
I am profoundly grateful for your contribution; you have not only implemented the integration of 3D Gaussian into the AIGC pipeline but also resolved the issue I previously perceived, where the performance of 3D Gaussian in AIGC was inferior to NeRF.
I have a query regarding a detail in the paper. It is mentioned in the paper that, to ensure the generated consistency, Point-E, a pre-trained large-scale model for text-to-point-cloud, is additionally utilized to optimize the position of point clouds in 3D Gaussians. My question is, how is the result of Point-E ensured to be consistent with the result of 2D Diffusion?
Looking forward to the author's reply!