oppo-us-research / SpacetimeGaussians

[CVPR 2024] Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis
https://oppo-us-research.github.io/SpacetimeGaussians-website/
Other
616 stars 45 forks source link

Question about Eq.8 and 9 #42

Open maincold2 opened 7 months ago

maincold2 commented 7 months ago

Thank you for sharing your nice work!

I have a question about the polynomial equations in the paper. Eq.8 (omitting i): mu(t) = sum_k{b_k(t - mu^tau)^k}, where b \in R.

But referring to the code below, I understand the formulation would be: mu(t) = mu + sum_k{b_k(t - mu^tau)^k}, where b \in R^3.

Although b_0 could be mu, I hardly understand why b is a scalar.

https://github.com/oppo-us-research/SpacetimeGaussians/blob/c04db700a49fe67c94cbd2a6cff2f7d9c518aba8/thirdparty/gaussian_splatting/renderer/__init__.py#L90

I wonder what I missed and your explanation, thanks!

lizhan17 commented 7 months ago

as tforpoly^0 is 1. the means3D at the right of equaiton is the b_i_0

 means3D = means3D*(tforpoly^0) +  pc._motion[:, 0:3] * tforpoly^1 + pc._motion[:, 3:6] * tforpoly ^2 + pc._motion[:, 6:9] * tforpoly ^3

there is a special case when tforpoly=0 (t=trbfcenter). as we can get the initial position of each point at its corresponding time (although the position generated by colmap may not be accurate).

a spacetime point can be obeserved at position means3D with highest possibility (trbf output is maximized) when t= trbfcenter.

maincold2 commented 7 months ago

Okay, I see, but the paper says b \in R, which I understand would be \in R^3. Would you please explain some about this?

lizhan17 commented 7 months ago

that should be a small typo. for 3-dimension data like position, R is R^3, for 4-d quaternion, R is R^4.