search

donydchen / mvsplat

🌊 [ECCV'24] MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images
https://donydchen.github.io/mvsplat
Other
498 stars 22 forks source link

Difference in diff-gaussian-rasterization-modified #6

Closed vidit98 closed 3 months ago

vidit98 commented 3 months ago

Hi, thanks for the great work. Can you please give some details what are the changes in the new diff-gaussian-rasterization package? Would the model trained with the new package compatible with old one?

donydchen commented 3 months ago

Hi @vidit98, thanks for your interest in our work.

The diff-gaussian-rasterization-modified is mainly modified to match the conventions of SH coefficients from e3nn with the one from the original diff-gaussian-rasterization. This is because MVSplat predicts SH coefficients in camera space, which need to be transformed into world space before rendering. To help with this transformation, we use Wigner D matrices from e3nn, so the conventions between the two packages need to be matched. And the model trained with the modified version is not compatible with the old one.

For more details, you can refer to this comment from the pixelSplat group.

vidit98 commented 3 months ago

Thank you very much for explaination. I checked the diff and if I understand correctly the only difference is in SH convention.

Just to better understand, if I train a model with diff-gaussian-rasterization-modified using only RGB it should be compatible with the diff-gaussian-rasterization

donydchen commented 3 months ago

In this case, I think you are correct. I have not tried it, though; you can give it a try.