search

hyperlogic / splatapult

A 3d gaussian splatting renderer in C++ and OpenGL
MIT License
92 stars 12 forks source link

Question for jacobian matrix #24

Open yuyaolong opened 2 weeks ago

yuyaolong commented 2 weeks ago

Hi Anthony, Do you have the derivation steps of this jacobian matrix or any material/links for better understanding? Thanks a lot ! https://github.com/hyperlogic/splatapult/blob/d05e3699384f82c7f12a8dfaadae244658aababe/shader/splat_vert.glsl#L167

yuyaolong commented 2 weeks ago

I thought the 3DGS still using ray space like in EWA volume paper. Looks I am wrong, it's using Projection matrix here

One more question: jsx should be -SX / t.z according to projection from eye coordinate to NDC here. what the * WIDTH / 2 here is for?

hyperlogic commented 2 weeks ago

If I remember correctly, the matrix in the paper wasn't quite the same projection transform used by OpenGL, also it did not include the viewport transformation. For the shader I needed something to go directly from view to viewport coords. In Mathematica I took the gradient of functions for each dimension (x, y, z) of the combined proj*viewport transform, including the homogeneous divide by z. And used this to end up with the Jacobian. image Which is what you see in the shader.

hyperlogic commented 2 weeks ago

About your second question, the (WIDTH / 2) term is from the "window to viewport" transform. My Jacobian includes both the projeciton matrix and the viewport transform.

yuyaolong commented 2 weeks ago

"window to viewport" transform means NDC [-1~1] -> window coordinates [0 ~ WinWidth]? If so, that makes sense to me since the direction vector(vec2 d = gl_FragCoord.xy - frag_p;) is in window coordinates too.