mkkellogg / GaussianSplats3D

Three.js-based implementation of 3D Gaussian splatting
MIT License
1.09k stars 134 forks source link

Issues rendering spherical harmonics #263

Open nickpaustian-slv opened 1 week ago

nickpaustian-slv commented 1 week ago

I have a very simple scene setup using the library via npm and it works perfectly, until I enable spherical harmonics.

Screenshot 2024-06-14 at 12 16 57 PM

I have tested the model in the online viewer (https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php) and it works there so I am sure it is something to do with my setup so I am still working through debugging, however I thought I would ask in case someone can point me in the right direction.

The code that loads the splat:

private async loadSplat() { this.splatViewer = new GaussianSplats3D.Viewer({ threeScene: this.scene, renderer: this.renderer, camera: this.camera, useBuiltInControls: false, sphericalHarmonicsDegree: 2, })

this.splatViewer
  .addSplatScene(plyUrl, {
    position: [0, 0, 0],
    rotation: composedRotation,
    scale: [100, 100, 100],
    progressiveLoad: false,
    showLoadingUI: false,
  })
  .then(() => {
    this.splatViewer.start()
  })

}

mkkellogg commented 1 week ago

Would you be willing to share your .ply file? Maybe I can do some troubleshooting on my end.

nickpaustian-slv commented 1 week ago

I believe I have uncovered the issue, it looks like the spherical harmonics are affected by the scale of the splat scene.

Here is the same splat at a lower scale:

Screenshot 2024-06-21 at 2 15 52 PM

and at a scale of 1:

Screenshot 2024-06-21 at 2 16 19 PM

I think I was using a higher scale to work with some other objects I had in my scene, but I should be able to scale those down to work with the splat instead.

mkkellogg commented 1 week ago

Thanks for figuring this out, it turns out I didn't take the scene's scale into account when rotating spherical harmonics (I just assumed a scale of <1, 1, 1>). I'll include a fix in the next release.

mkkellogg commented 4 days ago

I made an update in the 2dgs branch that should address this issue, want to try it out? You should be able to scale the scene now without affecting the spherical harmonics.

nickpaustian-slv commented 3 days ago

Thanks @mkkellogg, that is working great now when scaled.