search

Anttwo / SuGaR

[CVPR 2024] Official PyTorch implementation of SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering
https://anttwo.github.io/sugar/
Other
2.21k stars 166 forks source link

BEST BEST BEST one (NOT one-of) so far (almost all technical directions), but still a few challanges. #47

Open yuedajiong opened 10 months ago

yuedajiong commented 10 months ago

prioritized (object static mesh)：

  1. robust (the pose accuracy computed by COLMAP requirement is very high)
  2. quality (garbage point, connection, normals, ...)
  3. speed (2-hours level, goal is 10 minutes)
  4. vertex-face number (intermediate-state: 10-million-level, final mesh: 500K ; original mesh is just about 7k)
  5. better to have: the mask logic in algorithm native(GS and SuGaR parts) is necessary.

PS. another potential way: add-remove-move vertexes by gradients, and directly use nvdiffrast. (like GS operate gauss-points)

Anttwo commented 9 months ago

Hello @yuedajiong,

Thank you so much for you nice words! Indeed, these are very interesting ideas. Providing a mask functionality would be extremely useful for reconstructing synthetics scenes or scenes with texture-less background.

Concerning speed, as I explained in this issue, optimization takes 1h45\~1h50 only when you select the long refinement time option (which is the default setting). However, refinement only lasts a few minutes when using the short refinement time option. Please note that the short refinement is enough to provide a really good-looking hybrid representation in most cases. Currently, on average, the full optimization with short refinement takes 30\~35 min for coarse optimization and mesh extraction + a few additional minutes for short refinement. However, the code was built to be very modular, and splits all steps of the process into several scripts. As a consequence, we reload training data at the beginning of each step, which takes several minutes everytime. I think we can further reduce the optimization time just by avoiding such unnecessary reloading.

yuedajiong commented 9 months ago

for speed: 1) my GPU is titan-xp, very low-level. 2) better train policy. https://github.com/Anttwo/SuGaR/issues/14 (@Anttwo great-master, I think we can design a better and hpyer-parameters-less training-policy. not-only iteration, but also other necessary stat. data: loss-cure, point-number, gradients, ratio between your two optimization phases, ...) 3) my case, yes, 30 minitues are enough.