search

ingra14m / Deformable-3D-Gaussians

[CVPR 2024] Official implementation of "Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction"
https://ingra14m.github.io/Deformable-Gaussians/
MIT License
779 stars 40 forks source link

Question about applying the delta over scales and rotations #18

Closed XuyangBai closed 7 months ago

XuyangBai commented 7 months ago

Hi @ingra14m thank for sharing you great work! I have a question regarding the way you apply delta over gaussian scales and rotations, why did you apply the delta in the following ways

https://github.com/ingra14m/Deformable-3D-Gaussians/blob/7830e994632416b52f58f0f2dae5813ee8598960/gaussian_renderer/__init__.py#L87-L88

instead of applying delta before activation function, e.g.

scales = pc.scaling_activation(pc._scaling + d_scaling)
rotations = pc.rotation_activation(pc._rotation + d_rotation)

The way you implement is OK for scale but it does not guarantee a valid quaternion right?

ingra14m commented 7 months ago

Hi, thanks for your interest.

You are absolutely correct, the form you mentioned is absolutely correct physically. After going through the delta, quaternions should be normalized to unit quaternions to be meaningful. However, doing so in my experiments reduced the quality of rendering. You can think of this delta as a kind of trick.

Also, an interesting point is that I implemented quaternion multiplication in L20, which has more physical significance compared to quaternion addition. But unfortunately, the results are not as good as simply adding quaternions.

XuyangBai commented 7 months ago

I see, thanks for the explanation!