mkkellogg / GaussianSplats3D

Three.js-based implementation of 3D Gaussian splatting
MIT License
1.1k stars 135 forks source link

GPU Accelerated Sort Wrong on MacOS #199

Closed xiasun closed 2 months ago

xiasun commented 2 months ago

Hello all,

There's something wrong with GPU sort on my Mac.

When creating a Viewer with gpuAcceleratedSort set to true, the rendering result seems to be completely chaotic. However, if gpuAcceleratedSort is set to false or left undefined, everything turns out fine.

I noticed there was a commit to turn off GPU sort by default. Are there any known issues related to this problem?

Many thanks!

xiasun commented 2 months ago

To exclude the influence of SortWorker, I am using a pure JavaScript brute sort with pre-computed distances. However, the result doesn't change:

// brute sort something like this
sortedIndex.sort(function (a, b) { return distances[b] - distances[a]; });

So I guess the problem is within the webgl program to compute distances, but I truly cannot find anything suspecious. I also compared the sorted distance array value between windows and mac, most value are silimar, only some specific value are not same, and their value are not too strange.

mkkellogg commented 2 months ago

Interesting, I have tested the GPU accelerated sort on Mac OS and it works fine. Specifically I am using:

Processor: Apple M1 Pro
OS: MacOS Sonoma 14.2 (23C64)
Browser: Chrome 123.0.6312.124 (Official Build) (arm64)

I wonder if it's because if the particular GPU you are using. I turned off GPU accelerated sort by default because of reports that it didn't work correctly on some devices (possibly including yours). I think it needs to be tested on a larger variety of GPUs before I re-nable it by default.

Can you share all of your setup code so that I can try to reproduce the error? Also, what kind of GPU do you have?

xiasun commented 2 months ago

I am using the AMD Radeon Pro 5300M 4GB GPU on my Mac. The same problem can be reproduced on several Macs of the same model (but with different macOS and Chrome versions). It appears to be a compatibility problem specific to the older Mac models with an Intel CPU, AMD GPU, and x86_64 architecture.

The detail machine info is:

My setup code is pretty simple, just set a gpuAcceleratedSort field in truck.html, rebuild and visit the truck page

const viewer = new GaussianSplats3D.Viewer({
    'cameraUp': [0, -1, -.17],
    'initialCameraPosition': [-5, -1, -1],
    'initialCameraLookAt': [-1.72477, 0.05395, -0.00147],
    'gpuAcceleratedSort': true  // turn on gpu depth computation
});

BTW I just tested the gpuAcceleratedSort on a Mac with M1 Pro chip, and it looks fine.

xiasun commented 2 months ago

After a lot of comparing between windows and mac, I found out the problem is that TransformFeedback value wrong after index 65535 (start form zero). And can only be reproduced on Intel Mac with certain steps.

I have submitted a issue to google https://issuetracker.google.com/issues/333957180

mkkellogg commented 2 months ago

Thank you for looking into this! Do you know if the issue also occurs in Firefox or Safari?

xiasun commented 2 months ago

Glad to do this. This issue does not occur in Firefox or Safari.

xiasun commented 2 months ago

For anyone running into this problem to use GPU accelerated sorting on an Intel Mac, please switch to ANGLE's Metal backend as a temporary workaround.

mkkellogg commented 2 months ago

Good to know there's a workaround, thank you again for looking into this and filing the bug ticket!