search

mkkellogg / GaussianSplats3D

Three.js-based implementation of 3D Gaussian splatting
MIT License
1.23k stars 150 forks source link

v0.4.2, the processing splats are taking longer to process. #275

Open Masuji-Suto opened 1 month ago

Masuji-Suto commented 1 month ago

I have noticed that with v0.4.2, the processing splats are taking longer to process. Below is a test with a 3DGS .PLY file (270MByte) created with Luma AI. I used index.html and bonsai.html from DEMO. bonsai.html was measured by changing let path = 'demo.ply';. Processing splats time for bonsai.html (not including Download time) v0.4.1 : about 5 sec. v0.4.2 : approx. 38 sec.

index.html and displaying the same file, v0.4.1 : less than 1 second v0.4.2 : less than 1 second

In v0.4.2, what is the cause of the longer processing time for viewer.addSplatScene? index.html uses viewer.addSplatBuffers. viewer.addSplatScene also uses the same addSplatBuffers internally. Why is index.html so fast?

mkkellogg commented 4 weeks ago

That is definitely strange. For what it's worth, I can still load .ply files with version v0.4.2 quite quickly, not any slower than v0.4.1. Would you be willing to share your .ply file so I can debug?

Masuji-Suto commented 4 weeks ago

It's strange. I have tried it in various browsers and environments (Windows, Mac, iOS) and v0.4.2 is significantly slower than v0.4.1 in all environments except DEMO/index.html. The .ply is created by Luma AI and the iOS app Scaniverse. The .Ply used in this test is zipped and placed below. https://suto.bex.jp/tmp/demo.zip

Masuji-Suto commented 4 weeks ago

I found the cause. In v4.2.0, the default value of 'plyInMemoryCompressionLevel' was set to 2, so it was taking a long time. Setting it to 0 made it faster.

mkkellogg commented 4 weeks ago

Ah, that makes some sense. Although I don't why the in-memory compression is taking so long on your end when it seems pretty speedy on mine. It sounds like you've tried it on multiple machines & devices?

Masuji-Suto commented 4 weeks ago

The specifications of the PC on which the data was measured are as follows Windows11+Chrome 126.0.6478.127 (Official Build) 64bit Core i5-12400 8GB RAM GPU nVidia Quadro P400 Although not intended for gaming, it operates smoothly once the .PLY is loaded. I have not measured other PCs or iPhones, but they seem slower too.

As long as it is not compressed, there is no problem. Thank you for publishing this wonderful library. I have published the following web application using your library.

Display 3D Gaussian spreading with 3D images https://suto.bex.jp/lkg/gaus/index3de.html

3D Gaussian Splatting scene in Looking Glass https://suto.bex.jp/lkg/gaus/indexe.html

I am creating the iOS apps based on these web apps, but I am having problems with WKWebView on iOS reloading at random times. v0.4.2 did not help much either.

xiasun commented 2 weeks ago

Having a similar problem on iOS devices (iPad pro 4 and iPhone 13 with chrome).

The online truck page https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php can be loaded from iOS well. But the locally deployed truck page (of version v0.4.2 and v0.4.3) will stuck in processing splats and sometimes crash the browser.

I wonder is there any difference between the demo page and and the source of v0.4.2 or v0.4.3?

Thanks~

mkkellogg commented 2 weeks ago

Maybe I should make the plyInMemoryCompressionLevel parameter false by default? I didn't realize it would cause such serious performance issues. As for the online demo page, those all load .ksplat files, which are pre-processed and optimized from .ply files, and their content matches the internal representation my viewer uses to represent splat scenes in memory. That's why they load faster and require less memory when loading.

xiasun commented 2 weeks ago

It seems the latest truck.html source file also load ksplat file by default, does that mean the performance issue is not related with ply compression?