Weird rotation of splats in some section of point cloud

[kstoneriv3]

I have a relatively large .ply file (that was artificially generated from point cloud data). This file looks fine on another viewer (https://github.com/playcanvas/supersplat) but when I view it on the three.js viewer, the rotation of the splats are corrupted in some area. As can be seen in the screenshot, there seem to be a clear rectangular boundary which separates the area of corrupted rotation and one with fine rotation.

As the file was generated in a way that should not cause such an issue and the file looked fine on another viewer, I suspect there is some problem in the three.js viewer.

https://github.com/mkkellogg/GaussianSplats3D/assets/30857855/7a0610cc-fc78-4c85-98a1-d1eb82de016f

The boundary looks like a combination of surfaces of boxes, which makes me wonder if this has something to do with how points are stored and rendered within the viewer.

[kstoneriv3]

* I can share the .ply file for the debugging purpose, but I prefer not to share it publicly. Please let me know the gmail address so I can share the file via google drive. My email address is k.stoneriv3 [at] gmail.com.

[mkkellogg]

Yeah if you could share the .ply file as well as your code for setting up the viewer, that would be helpful. The best email to share it with would be mkkellogg [at] gmail.com

[kstoneriv3]

Hi @mkkellogg , that'd be great! Thank you! I should have shared the file on google drive (with my corporate email address). Please let me know if you cannot find the shared file.

As for the setting up the viewer, I just used the npm run demo and demo's "View a .ply, .ksplat, or .splat file" section with Camera up being 0, 0, 1 instead of 0, 1 ,0 of the default. All other parameters are the default values.

[mkkellogg]

So I'm having some trouble loading your .ply because it is too big :) Until I am able to get it loaded and debug it, you might try setting integerBasedSort: false for the Viewer constructor.

[kstoneriv3]

Thank you for checking this and sorry for the large file that sent you! I tried to trim down the data but with more aggressive trimming, but I could not reproduce this issue with the trimmed point cloud.

you might try setting integerBasedSort: false for the Viewer constructor.

I tried running the demo with integerBasedSort: false but it did not fix the problem.

[mkkellogg]

Out of curiosity, if you load your .ply into super splat and then export to .splat, and then load the .splat file using my viewer, does the problem still occur?

[kstoneriv3]

I still see the issue with the .splat file generated by supersplat but with some difference. This time, corrupted rotation appears in different section:

The red lines roughly indicate the boundary of corrupted splats and uncorrupted ones.

[mkkellogg]

So it turns out this issue is caused by a weird problem I've run into before where for some reason the maximum number of texels in a three.js data texture is 16777216 (2 ^ 24). I don't know what the root cause is, but for the time-being I have implemented a work-around by improving the way half-precision covariance values are packed into a texture. The work-around is now in the 2dgs branch and can be activated if you pass a value of true for the Viewer constructor parameter halfPrecisionCovariancesOnGPU. Can you try it out to see if it fixes the problem?

[kstoneriv3]

Thank you so much! I can confirm that your workaround works on my local environment as well.

The root cause you mentioned sounds greek to me but it is great to hear that it can be dealt with.

Do you have any ideas on when this fix becomes available on the main branch? (I am in no rush but just being curious...)

[mkkellogg]

Hopefully I will get this out as part of an official release within the next week.

[kstoneriv3]

Thank you! That's great to hear!!