Load a splat model stored on remote server?

[julesmorel]

Is it possible to load a .ply file stored on a remote location (a s3 bucket for instance)?

[mkkellogg]

If that remote location is CORS accessible then you should be able to load it using this library. For S3, I think you have to configure CORS accessibility: https://docs.aws.amazon.com/AmazonS3/latest/userguide/cors.html

[julesmorel]

Thank you for your fast reponse.

I configured CORS accessiblity as you advised, as you can see on the log below, it seems to be properly setup:

However, the promise splatBufferPromise resulting from loadFromFileData stays on pending, and is never resolved. Below is the function that loads the viewer, any hint would be greatly appreciated:

async function handleViewGaussianSplat() {

    try {
      const url = await getPresignedResultURLS3( ... );

      const viewerOptions = {
        'cameraUp': [0, -1, 0],
        'initialCameraPosition': [1, 0, 0],
        'initialCameraLookAt': [0, 1, 0],
        'halfPrecisionCovariancesOnGPU': false,
        'antialiased': false,
        'sphericalHarmonicsDegree': 0
      };

      const splatBufferOptions = {
        'splatAlphaRemovalThreshold': 1
      };

      const response = await fetch(url);
      console.log('Fetch response:', response);
      const arrayBuffer = await response.arrayBuffer();
      console.log('ArrayBuffer:', arrayBuffer);
      const format = GaussianSplats3D.LoaderUtils.sceneFormatFromPath(data.filename);
      console.log('File format:', format);

      let splatBufferPromise;
      if (format === GaussianSplats3D.SceneFormat.Ply) {
        splatBufferPromise = GaussianSplats3D.PlyLoader.loadFromFileData({ data: arrayBuffer }, 1, undefined, undefined, undefined, undefined, 1);
      } else if (format === GaussianSplats3D.SceneFormat.Splat) {
        splatBufferPromise = GaussianSplats3D.SplatLoader.loadFromFileData({ data: arrayBuffer }, 1, undefined, undefined, undefined, undefined, 1);
      } else {
        splatBufferPromise = GaussianSplats3D.KSplatLoader.loadFromFileData({ data: arrayBuffer });
      }

      console.log('splatBufferPromise:', splatBufferPromise);

      splatBufferPromise.then((splatBuffer) => {
        console.log('Splat Buffer Loaded:', splatBuffer);
        document.getElementById("demo-content").style.display = 'none';
        document.body.style.backgroundColor = "#000000";

        let viewer = new GaussianSplats3D.Viewer(viewerOptions);
        viewer.addSplatBuffers([splatBuffer], [splatBufferOptions])
          .then(() => {
            console.log('Starting viewer');
            viewer.start();
          }).catch((error) => {
            console.error('Error starting viewer:', error);
          });
      }).catch((error) => {
        console.error('Error loading splat buffer:', error);
      });

      onClose();
    } catch (error) {
      console.error('Error in handleViewGaussianSplat:', error);
    }
  };

[mkkellogg]

One thing that I'm seeing is that in your code you are passing { data: arrayBuffer } as the first argument to loadFromFileData() -- I think you want to pass arrayBuffer as the first argument instead.

[julesmorel]

It works now after setting 'sharedMemoryForWorkers': false in the viewerOptions.

Is there a way to use the DropInViewerwith the splatBuffer?

[mkkellogg]

There is, and I'm actually surprised you're loading splats the way you are, there's definitely an easier way! You can use the function Viewer.addSplatScene() or DropInViewer.addSplatScene() to load any scene type:

const url = await getPresignedResultURLS3(user.sub, idToken, region, idpool, `${data.filename}`, 86400);
const format = GaussianSplats3D.LoaderUtils.sceneFormatFromPath(data.filename);

const viewer = new GaussianSplats3D.Viewer({
  'cameraUp': [0, -1, 0],
  'initialCameraPosition': [1, 0, 0],
  'initialCameraLookAt': [0, 1, 0],
  'halfPrecisionCovariancesOnGPU': false,
  'antialiased': false,
  'sphericalHarmonicsDegree': 0
});
viewer.addSplatScene(url, {
    'format': format,
    'splatAlphaRemovalThreshold': 1,
})
.then(() => {
    viewer.start();
});

[julesmorel]

Thank you again for your help. Actually I would prefer to go the way you propose but it does not seem to work. If I use the DropInViewer this way

  // Create a scene
  const scene = new THREE.Scene();
  scene.background = new THREE.Color(0xeeeeee);

  // Create a camera
  const camera = new THREE.PerspectiveCamera(75, mount.clientWidth / mount.clientHeight, 0.1, 1000);
  camera.position.z = 5;

  // Create a renderer
  const renderer = new THREE.WebGLRenderer();
  renderer.setSize(mount.clientWidth, mount.clientHeight);
  mount.appendChild(renderer.domElement);
  rendererRef.current = renderer;

  const viewer = new GaussianSplats3D.DropInViewer({
      'cameraUp': [0, -1, 0],
      'initialCameraPosition': [1, 0, 0],
      'initialCameraLookAt': [0, 1, 0],
      'halfPrecisionCovariancesOnGPU': false,
      'antialiased': false,
      'sphericalHarmonicsDegree': 0,
      'sharedMemoryForWorkers': false
  });
  viewer.addSplatScene(url, {
      'format': 2,
      'splatAlphaRemovalThreshold': 1,
  });
  scene.add(viewer);

  // Animation loop
  const animate = () => {
      if (rendererRef.current) {
          requestAnimationFrame(animate);
          renderer.render(scene, camera);
      }
  };

  animate();

Then I encounter the following error :

ERROR End of file reached while searching for end of header extractHeaderFromBufferToText@http://localhost:3000/static/js/bundle.js:197367:15 determineHeaderFormatFromPlyBuffer@http://localhost:3000/static/js/bundle.js:197424:40 parseToUncompressedSplatArray@http://localhost:3000/static/js/bundle.js:197841:38

Something interesting, the error changes if I remove 'format': 2 from the options:

ERROR AbortablePromise.reject is not a function downloadSplatSceneToSplatBuffer@http://localhost:3000/static/js/bundle.js:204623:29 downloadAndBuildSingleSplatSceneStandardLoad@http://localhost:3000/static/js/bundle.js:204429:34 addSplatScene@http://localhost:3000/static/js/bundle.js:204412:12 addSplatScene@http://localhost:3000/static/js/bundle.js:205475:24 initializeScene@http://localhost:3000/main.9b93db0cc54f5f187d55.hot-update.js:69:14

[mkkellogg]

My guess is that there is something wrong with the format of your .ply file, would you be willing to share it? As for the error you see when you remove 'format': 2, that's a bug and it really should be displaying an error that the file format is not supported. I think it's happening here: https://github.com/mkkellogg/GaussianSplats3D/blob/19e01ad9455ae56a5c4b141a1101a6e3baf90750/src/Viewer.js#L1024 The problem is AbortablePromise doesn't have a reject() function :). I will fix that.

[julesmorel]

Sure, here is the file.

But it loads properly on your demo. And also when I am using the splatBuffer and addSplatBuffers like in the piece of code I posted few comments above.

[mkkellogg]

There must be some issue with the format of the file you're retrieving from S3. The error you saw earlier, End of file reached while searching for end of header, happens when the library tries to parse a file as a .ply, but can't find the end_header token that should be in the file.

[julesmorel]

You are right the problem was due to an error on my side. I apologize for any inconvenience caused. I'll go ahead and close this issue.

[mkkellogg]

Glad you got it sorted out!