search

dcharatan / pixelsplat

[CVPR 2024 Oral, Best Paper Runner-Up] Code for "pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction" by David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann
http://davidcharatan.com/pixelsplat/
MIT License
864 stars 60 forks source link

Confusion about spherical harmonic coefficients #51

Closed caiyongqi closed 7 months ago

caiyongqi commented 7 months ago

Is the following code transforming the predicted SH coefficients in the camera coordinate system to the world coordinate system?

https://github.com/dcharatan/pixelsplat/blob/f4ae9c1396ce62097b407e819321f5819d7788a9/src/model/encoder/common/gaussian_adapter.py#L90

In addition, the "diff-gaussian-rasterization-modified" only modifies the content related to the SH coefficient and has no effect on other parameters(scale, rotate...)?

dcharatan commented 7 months ago

Yes, it's transforming the SH coefficients from the camera coordinate system to the world coordinate system. The diff-gaussian-rasterization-modifed code changes the convention for the spherical harmonics to match the e3nn library's conventions. This makes it possible to use the Wigner D matrices provided by e3nn to rotate the spherical harmonics. For more details, see this comment.