Spherical harmonics get corrupted farther from the origin

[TheExDeus]

If the splat is large, then spherical harmonics get all funky after a constant distance.

These distance isn't radial though - it starts at some constant offset in world space (so a cube around origin in okay and everything outside of it has messed up sh):

This example is with ksplat compression 1, but it's same with PLY without compression. Doesn't happen if I specify 0 or 1 harmonic level or viewer, only 2 or 3.

I will try investigating what could cause this, as my initial guess of compression isn't right. I'm using 0.4.1 version. I can send the PLY/ksplat directly.

[mkkellogg]

I have also seen issues with spherical harmonics that only seem to occur on large splats, and I wonder if it's an issue with either the browser or graphics driver not handling large textures correctly. Would you be able to share the .PLY file? Maybe I can debug it.

[mkkellogg]

So based on my observations, it looks like I just can't use textures for spherical harmonics that are larger than 4096x4096, or have a width or height that is greater than 4096, regardless of the texel format (RG, RGBA, etc.). Or I can but only the texture data below the 4096x4096 boundary (16777216 texels) seems to be valid. So instead of spending more time trying to understand what exactly is happening, I am just splitting the spherical harmonics data into 3 textures in that case (one for each color channel) 😄 I have a prototype implementation in this branch if you want to try it out: https://github.com/mkkellogg/GaussianSplats3D/tree/general_updates.

[TheExDeus]

Yup, it works now, thank you! Though you use textures to pack because it's more compact or performant than if you use uniforms? Because you still calculate the resulting color in shader, so the harmonics could be passed as uniform sh_type sh[max_sh] per gaussian? This way there won't be a texture limit, but not sure about how big can uniforms be. Could be packed in Uniform Buffer Objects too.

[mkkellogg]

It might be technically possible to store the SH data in uniforms, but it would be a lot trickier to both store them and access them in the shader, given that the maximum size of a single uniform block is small (65536 on my machine). Not only that, but I have no idea if caching of uniforms access is even a thing, but texture data caching definitely is, which boosts performance when data is laid out in the texture strategically.

[TheExDeus]

Yeah, multiple blocks would be needed. So with textures the current maximum amount of splats is 4096*4096=16777216 right? That would be 256 uniform blocks on your GPU. I guess SSBO's would be an alternative, but not sure it's faster than textures, though then it's technically only limited by GPU ram.

And access would be a bit tricky, but you already have splatIndexes which could help.

[mkkellogg]

FYI this fix is now in main :)