search

zju3dv / PGSR

[TVCG2024] PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction
https://zju3dv.github.io/pgsr/
Other
507 stars 33 forks source link

Training Speed Compared to 3DGS #51

Closed GuangyuWang99 closed 1 month ago

GuangyuWang99 commented 1 month ago

Hi, thanks for the excellent work!

I find the optimization of PGSR seems to be ~3x slower than the original 3DGS. To take a closer look, I have disabled the multi-view terms and found the bottleneck is likely to come from the calculation of the single-view term (e.g., plane rasterization of other attributes, depth_normal calculation, or the backward flow through them). So what operation(s) do you think to be the main bottleneck?

GuangyuWang99 commented 1 month ago

Does your adaptive density control tend to densify gaussians more aggressively than the original 3DGS's?

danpeng2 commented 1 month ago

Hi, compared to 3D Gaussian Splatting (3DGS), we have slightly more Gaussian points, which may affect the training speed. However, based on our observations, this is not the main bottleneck in most common scenarios. With the same number of Gaussian points, there are two potential ways to improve training speed:

  1. As you mentioned, the bottleneck comes from the backpropagation of the plane parameters. Currently, the backpropagation of the plane parameters in the code is not implemented using CUDA. If CUDA is used for backpropagation, the training speed could be significantly improved.

  2. Merging multi-view rendering. The reference frame and neighboring frames often share a large number of co-visible Gaussians. If their operations can be merged in CUDA, it could also potentially improve training speed.