graphdeco-inria / diff-gaussian-rasterization

Other
827 stars 254 forks source link

question about gradients in 'computeCov2DCUDA' in backward.cu #59

Open Choujim opened 3 weeks ago

Choujim commented 3 weeks ago

there are some confusing caculations of gradients, i try to understand it, maybe i need some help: (1)If gradients is calculated according to 'cov2D = transpose(T) transpose(Vrk) T;' some T_ij seems to be aliasing, such as T[1][0] should be T[0][1] .

// Gradients of loss L w.r.t. each 3D covariance matrix (Vrk) entry, // given gradients w.r.t. 2D covariance matrix (diagonal). // cov2D = transpose(T) transpose(Vrk) T; dL_dcov[6 idx + 0] = (T[0][0] T[0][0] dL_da + T[0][0] T[1][0] dL_db + T[1][0] T[1][0] dL_dc); dL_dcov[6 idx + 3] = (T[0][1] T[0][1] dL_da + T[0][1] T[1][1] dL_db + T[1][1] T[1][1] dL_dc); dL_dcov[6 idx + 5] = (T[0][2] T[0][2] dL_da + T[0][2] T[1][2] dL_db + T[1][2] T[1][2] * dL_dc); ..........

(2)Also with followings:

// Gradients of loss w.r.t. upper 2x3 portion of intermediate matrix T // cov2D = transpose(T) transpose(Vrk) T;
float dL_dT00 = 2 (T[0][0] Vrk[0][0] + T[0][1] Vrk[0][1] + T[0][2] Vrk[0][2]) dL_da + (T[1][0] Vrk[0][0] + T[1][1] Vrk[0][1] + T[1][2] Vrk[0][2]) * dL_db;
..........

(3)with W[0][1] and so on

// Gradients of loss w.r.t. upper 3x2 non-zero entries of Jacobian matrix // T = W J float dL_dJ00 = W[0][0] dL_dT00 + W[0][1] dL_dT01 + W[0][2] dL_dT02; ........

wongshek commented 3 weeks ago

I also raised the same issue. It seems that the gradient calculation here used the same formula as in the paper: $J W Σ W^\top J^\top$, but the cov2D calculation code used: $J^\top W^\top Σ^\top W J$

Choujim commented 3 weeks ago

I also raised the same issue. It seems that the gradient calculation here used the same formula as in the paper: J W Σ W ⊤ J ⊤ , but the cov2D calculation code used: J ⊤ W ⊤ Σ ⊤ W J

Thank you! You are right that since W=R_CW(w2c) in code, $JW \Sigma W^TJ^T$ should be the corresponding format as EWA paper. I've also found that $\Sigma$ used in 'computeCov3D' is $R^TS^TSR$ , not the $RSS^TR^T$ as the paper too. So the actual format in code is $(J^TW^T)R^TS^TSR(WJ)$ . What does it really means hhh,have you any idea about that?