mkkellogg / GaussianSplats3D

Three.js-based implementation of 3D Gaussian splatting
MIT License
1.54k stars 199 forks source link

v0.4.2, the processing splats are taking longer to process. #275

Closed Masuji-Suto closed 2 months ago

Masuji-Suto commented 4 months ago

I have noticed that with v0.4.2, the processing splats are taking longer to process. Below is a test with a 3DGS .PLY file (270MByte) created with Luma AI. I used index.html and bonsai.html from DEMO. bonsai.html was measured by changing let path = 'demo.ply';. Processing splats time for bonsai.html (not including Download time) v0.4.1 : about 5 sec. v0.4.2 : approx. 38 sec.

index.html and displaying the same file, v0.4.1 : less than 1 second v0.4.2 : less than 1 second

In v0.4.2, what is the cause of the longer processing time for viewer.addSplatScene? index.html uses viewer.addSplatBuffers. viewer.addSplatScene also uses the same addSplatBuffers internally. Why is index.html so fast?

mkkellogg commented 4 months ago

That is definitely strange. For what it's worth, I can still load .ply files with version v0.4.2 quite quickly, not any slower than v0.4.1. Would you be willing to share your .ply file so I can debug?

Masuji-Suto commented 4 months ago

It's strange. I have tried it in various browsers and environments (Windows, Mac, iOS) and v0.4.2 is significantly slower than v0.4.1 in all environments except DEMO/index.html. The .ply is created by Luma AI and the iOS app Scaniverse. The .Ply used in this test is zipped and placed below. https://suto.bex.jp/tmp/demo.zip

Masuji-Suto commented 4 months ago

I found the cause. In v4.2.0, the default value of 'plyInMemoryCompressionLevel' was set to 2, so it was taking a long time. Setting it to 0 made it faster.

mkkellogg commented 4 months ago

Ah, that makes some sense. Although I don't why the in-memory compression is taking so long on your end when it seems pretty speedy on mine. It sounds like you've tried it on multiple machines & devices?

Masuji-Suto commented 4 months ago

The specifications of the PC on which the data was measured are as follows Windows11+Chrome 126.0.6478.127 (Official Build) 64bit Core i5-12400 8GB RAM GPU nVidia Quadro P400 Although not intended for gaming, it operates smoothly once the .PLY is loaded. I have not measured other PCs or iPhones, but they seem slower too.

As long as it is not compressed, there is no problem. Thank you for publishing this wonderful library. I have published the following web application using your library.

Display 3D Gaussian spreading with 3D images https://suto.bex.jp/lkg/gaus/index3de.html

3D Gaussian Splatting scene in Looking Glass https://suto.bex.jp/lkg/gaus/indexe.html

I am creating the iOS apps based on these web apps, but I am having problems with WKWebView on iOS reloading at random times. v0.4.2 did not help much either.

xiasun commented 4 months ago

Having a similar problem on iOS devices (iPad pro 4 and iPhone 13 with chrome).

The online truck page https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php can be loaded from iOS well. But the locally deployed truck page (of version v0.4.2 and v0.4.3) will stuck in processing splats and sometimes crash the browser.

I wonder is there any difference between the demo page and and the source of v0.4.2 or v0.4.3?

Thanks~

mkkellogg commented 4 months ago

Maybe I should make the plyInMemoryCompressionLevel parameter false by default? I didn't realize it would cause such serious performance issues. As for the online demo page, those all load .ksplat files, which are pre-processed and optimized from .ply files, and their content matches the internal representation my viewer uses to represent splat scenes in memory. That's why they load faster and require less memory when loading.

xiasun commented 4 months ago

It seems the latest truck.html source file also load ksplat file by default, does that mean the performance issue is not related with ply compression?

mkkellogg commented 2 months ago

FWIW, in the memory-optimizations branch I have made some improvements to the memory usage of the viewer, as well as made the in-memory compression of non-ksplat files disbaled by default

mkkellogg commented 2 months ago

In the latest version (v0.4.5), the default compression level is back to being 0.