search

mkkellogg / GaussianSplats3D

Three.js-based implementation of 3D Gaussian splatting
MIT License
1.53k stars 198 forks source link

.KSplat Color issue with spherical harmonic #375

Open hyqshr opened 1 week ago

hyqshr commented 1 week ago

While rendering my scene with .ksplat with 0 sphericalHarmonicsDegree, it worked well:

        const viewer = new GaussianSplats3D.DropInViewer({
            sharedMemoryForWorkers: false,
            showLoadingUI: true,
            format: GaussianSplats3D.SceneFormat.KSplat,
            sphericalHarmonicsDegree: 0
        });
image

But when I change the sphericalHarmonicsDegree to 1 or 2, the color seems to be broken:

image
mkkellogg commented 5 days ago

Out of curiosity, how many splats are in your scene?

hyqshr commented 5 days ago

Only 1 splat.

The broken color issue happens on .ply as well, but only when sh_degree is 1 (0 or 2 is fine). My model is trained from splatfacto-w (https://docs.nerf.studio/nerfology/methods/splatw.html) not sure if this the reason?

mkkellogg commented 5 days ago

Sorry let me rephrase, how many individual splat "blobs" are in your scene? If you press the 'I" key, you'll bring up an info panel that will give you that info:

image

hyqshr commented 5 days ago

I press "I" and I can not bring up the info panel. Is there other way I could print this?

hyqshr commented 5 days ago

Also - is there a python script to convert ply to ksplat? I think it can help a lot of folk's workflow. I can create a PR if needed

mkkellogg commented 4 days ago

If you want to send me your .ksplat or .ply file, I can tell you how many splats it has. My suspicion is that it has a large number, which can cause issues because there's not enough memory to store spherical harmonics data.

You could also load it in the online viewer here: https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php and if you press 'I' in that viewer, it should bring up the info panel.

hyqshr commented 4 days ago

Below is the info panel. The weird thing is the broken color seems only happens at sh_degree=1 for this KSplat. 0 and 2 are fine

image
hyqshr commented 4 days ago

I have another model have the same issue when sh_degree is 1 in viewer

image
mkkellogg commented 4 days ago

That is pretty weird, would you mind sharing your .ply or .ksplat file and I can try to debug?

hyqshr commented 4 days ago

Thank you! Not sure why this link download .ttf format, but I can reproduce the error when I change it to .ksplat

https://forreal.blob.core.windows.net/forreal-model/0/splat-w-office.ksplat?sv=2023-01-03&st=2024-11-22T16%3A33%3A19Z&se=2024-11-23T16%3A33%3A19Z&sr=b&sp=r&sig=YVWvuRTBzwTA4UgOh%2BPyn5UQ9sKhU8EC01dmI4idyr4%3D

mkkellogg commented 3 days ago

Would you also be able to share your original .ply as well? And do you know how many SH degrees should be present in the file?

Also, I get an error when I try to access the above link:

image