mkkellogg / GaussianSplats3D

Three.js-based implementation of 3D Gaussian splatting
MIT License
1.09k stars 134 forks source link

Model rendering does not work in browsers on iOS versions below 16.4 #241

Closed whwhwwhwwh closed 3 weeks ago

whwhwwhwwh commented 1 month ago

Hi,

First of all, I want to say that your model library is fantastic, and I really enjoy using it. However, I have encountered an issue where the model can not rendering on iPhones running iOS versions below 16.4.

Could you please look into this issue and let me know what might be causing it?

Thank you!

mkkellogg commented 1 month ago

I'll see if I can find a device with an appropriate iOS version and I'll get back to you :)

whwhwwhwwh commented 1 month ago

I reproduced this issue on an iOS 16.1 simulator, as shown in the screenshots below (Figures 1 and 2 are the exception captures I added in the source code, and Figure 3 is the console error). The error occurs in the WebAssembly.compile(sorterWasmBytes) section. Could you take a look and see how this issue can be resolved?

note1

note2 error

mkkellogg commented 3 weeks ago

I should have asked this earlier, but does this error occur only in specific browsers (like Safari) or all browsers?

mkkellogg commented 3 weeks ago

So after a little experimentation, it looks like this bug is caused by the usage of SIMD instructions in the web assembly module for sorting splats. If I remove all SIMD code and recompile without the -msimd128 flag, I can get it to work on an iOS simulator with iOS version 16.1. I guess the best solution is to include both kinds of sorting modules in the build and detect the iOS version to determine which one to use.

mkkellogg commented 3 weeks ago

I made an update that forces a non-SIMD version of the splat sort when using iOS < 17.0 here: https://github.com/mkkellogg/GaussianSplats3D/tree/general_updates. Would you be able to try that out?

whwhwwhwwh commented 3 weeks ago

Thank you very much for your response and the provided solution. I tested it on an iOS 16.1 simulator and it works fine under normal circumstances. However, when I set sharedMemoryForWorkers to false or when I render the model using an iOS 15.0 simulator, I encounter the following error: "WebAssembly.Module doesn't parse at byte 67shared memory is not enabled". I am not sure how to resolve this issue. Could you please take another look?

WechatIMG44287

mkkellogg commented 3 weeks ago

Ok I've made a few more tweaks targeting iOS versions below 16.0 to that same branch, want to give it another try?

whwhwwhwwh commented 3 weeks ago

I tested the branch version you provided on devices running iOS 15 and iOS 16. There were no errors, and it rendered successfully! Haha, thanks a lot!

mkkellogg commented 3 weeks ago

Great, glad it worked!

whwhwwhwwh commented 4 days ago

Hello, after conducting my tests, there still seems to be an issue with iOS versions below 16.4. I think the following changes can be made to the code in the file /src/worker/SortWorker.js:

    // iOS makes choosing the right WebAssembly configuration tricky :(
    let iOSSemVer = isIOS() ? getIOSSemever() : null;
    if (!enableSIMDInSort && !useSharedMemory) {
        sourceWasm = SorterWasmNoSIMD;
        if (iOSSemVer && iOSSemVer.major < 16) {
            sourceWasm = SorterWasmNoSIMDNonShared;
        }
    } else if (!enableSIMDInSort) {
        sourceWasm = SorterWasmNoSIMD;
    } else if (!useSharedMemory) {
        if (iOSSemVer && iOSSemVer.major < 16) {
            sourceWasm = SorterWasmNonShared;
        }
    }

In the code, change the condition if (iOSSemVer && iOSSemVer.major < 16) { to if (iOSSemVer && iOSSemVer.major <= 16 && iOSSemVer.minor < 4) Moreover, according to the previous logic, the if inside the last else if should not be reachable.

Therefore, the modified code segment would be:

  // iOS makes choosing the right WebAssembly configuration tricky :(
  const iOSSemVer = isIOS() ? getIOSSemever() : null
  if (!enableSIMDInSort && !useSharedMemory) {
    sourceWasm = SorterWasmNoSIMD
    if (iOSSemVer && iOSSemVer.major <= 16 && iOSSemVer.minor < 4)
      sourceWasm = SorterWasmNoSIMDNonShared
  }
  else if (!enableSIMDInSort) {
    sourceWasm = SorterWasmNoSIMD
  }