Can anybody help me explain the origin of formula 6?

[yuedajiong]

The authors said that the gradient-descent can not directly optimize out the VALID covariance matrix likes formula #5, and they designed the formula #6.

Can anybody provide more explanations?

[kwea123]

it's not formula 5 exactly, they only refer to $\Sigma$. The naive way to parametrize this matrix is by setting 6 numbers (the upper triangle since this matrix is symmetric) and optimize them independently. But the problem is that as the optimization goes, this matrix is not guaranteed to be postive semi-definite (which it must be because it's a covariance matrix). Therefore, to guarantee that it is always positive semi-definite no matter how the gradient descent works, they design formula 6. By definition such matrix (in the form of $A^TA$) is always positive semi-definite. This is a trick we call reparametrization which is often used to avoid unexpected outcome of the optimization.

[yuedajiong]

oh my god, @kwea123, great-master!!! I have heard your technical sharing about instant-npg, from a mainland site named bilibili.com.

thanks !!!

and read this link: https://en.wikipedia.org/wiki/Multivariate_normal_distribution, understood:

logic, step by step: (representation)

1. by contrast to continuous nerf and descrete point-cloud, GS want to use parameterization to improve deterministic simple point(xyz +c) to probabilistic representation (xyz+G +c+...), and the distribution used Gauss （good match property).
2. there are 3 dimensions to be described, so, this is a multivariate normal distribution. by definition, X ~ N(miu, sigma), so, we need to define miu and sigma, in GS, the miu is defined as mean3D, somewhere named xyz in code. then, how to define sigma for multivariate? simply, refer to 'Equivalent definitions' in wiki, and formula 4 in paper: a symmetric, positive semidefinite matrix sigma. to keep the P-SD property, we can 'prove' and decompose S into S=U_up_t U. (To Chinese friends, please refer to https://zhuanlan.zhihu.com/p/268296897). And further, because sigma is an anellipsoid (math image, or https://en.wikipedia.org/wiki/Ellipsoid), we can decompose it in to sigma S = Scale Rotation. Finally, merge all, Sigma = U_up_t U, Sigma = Scale Rotation -> Sigma = S_up_t S R_up_t * R, this is formula 6.
3. formula 5 E'/sigma' is projection in camera 2d space.