mkkellogg / GaussianSplats3D

Three.js-based implementation of 3D Gaussian splatting
MIT License
1.09k stars 134 forks source link

pick up intersection points using ray #250

Closed zsjssj closed 2 weeks ago

zsjssj commented 3 weeks ago

I want to pick up intersection points using ray like follows

  window.addEventListener('mousedown', (e) => {
    if (e.button === 1) {
      const raycaster = new THREE.Raycaster()
      pointer.x = (e.clientX / container.clientWidth) * 2 - 1
      pointer.y = -(e.clientY / container.clientHeight) * 2 + 1
      raycaster.setFromCamera(pointer, camera)
      if (scene) {
        // let points = scene.children.find((item) => item.isPoints)

        console.log('viewer,scene', e, viewer, scene) //scene.children
        const intersects = raycaster.intersectObjects(viewer.children)
        const geometry = new THREE.BoxGeometry(0.01, 0.01, 0.01)
        const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 })
        const cube = new THREE.Mesh(geometry, material)
        // points.raycast(raycaster, intersects)
        console.log('intersects', intersects[0], intersects)
        // let intersects1 = intersects.filter((item) => item.object.isPoints && item.distance >= 0.00001)
        // console.log('intersects1', intersects1)
        intersects[0] && cube.position.copy(intersects[0].point)
        scene.add(cube)
      }
    }
  })

But I can only get intersection points in a invisible sphere at the origin like in the following image. And it is not possible to get intersect point outside the sphere area. image

mkkellogg commented 2 weeks ago

Ah, right now raycasting with a splat mesh doesn't work with three.js' raycaster, because raycasting against a splat mesh works differently (it returns intersections with splats, not three.js meshes).

Here is an example of how to use the raycaster that I made for this library specifically for raycasting against splats:

const viewer = new GaussianSplats3D.Viewer();
viewer.addSplatScene(path)
.then(() => {
   viewer.start();
   viewer.renderer.domElement.addEventListener('pointerdown', onMouseClick, false);
});

const onMouseClick = function(mouse) {
  raycastSplatMesh(mouse.offsetX, mouse.offsetY);
}

const raycastSplatMesh = function() {

    const renderDimensions = new THREE.Vector2();
    const toNewFocalPoint = new THREE.Vector3();
    const mousePos = new THREE.Vector3();
    const outHits = [];

    return function(x, y) {
        viewer.getRenderDimensions(renderDimensions);
        mousePos.set(x, y);
        outHits.length = 0;
        viewer.raycaster.setFromCameraAndScreenPosition(viewer.camera, mousePos, renderDimensions);
        viewer.raycaster.intersectSplatMesh(viewer.splatMesh, outHits);
        if (outHits && outHits.length > 0) {
          console.log(outHits[0])
        }
    };

}();
zsjssj commented 2 weeks ago

This solves my problem, thank you.

mkkellogg commented 2 weeks ago

Great, glad that fixed it!