Picking in a PointCloud with variable size points

[mikecantor]

Thanks so much for adding PointCloud support for RayCasting in r.61! Unless I am missing something, the current implementation assumes that the points are all of the same size. Is it possible to make the raycaster aware of the gl_PointSize of each point, rather than uniformly using params.PointCloud.threshold?

Thanks, Mike Cantor

[mikecantor]

Looking at the code, this might be as simple as giving allowing for a new optional param on raycaster,

raycaster.params.PointCloud.thresholds //(plural)

where thresholds is an array with the same size as the number of points, and these are used on a per-point basis to compare to rayPointDistance.

[danpaulsmith]

+1 I'm using a PointCloud of different sized particles. Trying to figure out how I can make the threshold parameter dynamic. @mikecantor did you have success with the "thresholds" approach?

[mikecantor]

Hi Dan,

Yes, I got it working by modifying the ray-caster code. It's not super pretty but it works. In my case I I have an additional complication that I am using an orthographic cam and navigating the scene by setting the top/left/bottom/right of the cam so my points end up being ellipses in the space that the ray-caster sees. Anyway, the code is below -- this is a modification of the THREE.PointCloud.prototype.raycast function in three.js (r68)

I add five new properties to the raycaster.params: PointCloud.pointSizes, screenWidth, screenHeight, camWidth, and camHeight.

THREE.PointCloud.prototype.raycast = ( function () {

    var inverseMatrix = new THREE.Matrix4();
    var ray = new THREE.Ray();
        var pointToStr = function(point) {
            return point.x + "\t" + point.y + "\t" + point.z;
        };

    return function ( raycaster, intersects ) {

            var object = this;

            var geometry = object.geometry;
            var pointSizes = raycaster.params.PointCloud.pointSizes;

            var screenWidth = raycaster.params.screenWidth;
            var screenHeight = raycaster.params.screenHeight;
            var camWidth = raycaster.params.camWidth;
            var camHeight = raycaster.params.camHeight;

            var xPerPixel = camWidth / screenWidth;
            var yPerPixel = camHeight / screenHeight;

            inverseMatrix.getInverse( this.matrixWorld );
            ray.copy( raycaster.ray ).applyMatrix4( inverseMatrix );

            if ( geometry.boundingBox !== null ) {

        if ( ray.isIntersectionBox( geometry.boundingBox ) === false ) {

            return;

        }
            }
            var position = new THREE.Vector3();

            var testPoint = function ( point, index ) {

                var intersectPoint = ray.closestPointToPoint( point );
                var pixelRadius = pointSizes[index]/2;

                // We must determine if intersectPoint is within an elipse that is centered on "point"
                // and with r1 and r2 calcuated as follows : 

                var rx = pixelRadius * xPerPixel ;
                var ry = pixelRadius * yPerPixel ;

                var x1 = intersectPoint.x;
                var y1 = intersectPoint.y;
                var x2 = point.x;
                var y2 = point.y;

                var inElipse = (Math.pow(x1 - x2, 2) / Math.pow(rx, 2)) + (Math.pow(y1 - y2, 2) / Math.pow(ry, 2)) <= 1;

                if (inElipse) {

                    intersectPoint.applyMatrix4(object.matrixWorld);

                    var rayPointDistance = ray.distanceToPoint( point ); 
                    var distance = raycaster.ray.origin.distanceTo(intersectPoint); 

                    intersects.push({
                        distance: distance,
                        distanceToRay: rayPointDistance,
                        point: intersectPoint.clone(),
                        index: index,
                        face: null,
                        object: object

                    });

                }

            };

I call this function as follows:

        raycaster.params.PointCloud.pointSizes = pointsManager.pointSizes;
        raycaster.params.camWidth = camera.right - camera.left;
        raycaster.params.camHeight = camera.top - camera.bottom;        
        raycaster.params.screenWidth = plotWidth;
        raycaster.params.screenHeight = plotHeight;

        var mouseX = (containerX / renderer.domElement.clientWidth) * 2 - 1;
        var mouseY = -(containerY / renderer.domElement.clientHeight) * 2 + 1;

        var rayOrigin = new THREE.Vector3(mouseX, mouseY, 0);
        projector.unprojectVector(rayOrigin, camera);
        rayOrigin.z = PlotConfig.MAX_Z + 1;
        raycaster.ray.set( rayOrigin, new THREE.Vector3(0,0,-1) );
        var intersections = raycaster.intersectObjects( [pointCloud]); //pointCloud is a THREE.PointCloud

[sasha240100]

/ping @mikecantor @danpaulsmith

Any progress with it in three.js core?

I wrote my own implementation for r86 of testPoint function:

function testPoint( point, index ) {
    var rayPointDistanceSq = ray.distanceSqToPoint( point );

       // THE ALGORITHM STARTS HERE

    var camPlane = new THREE.Plane(object.cam.position.clone().normalize()); // plane looking towards the camera
    var inVec = ray.intersectPlane(camPlane); // point of camPlane & ray intersection
    var pointRay = new THREE.Ray(ray.origin, point.clone().sub(ray.origin).normalize()) // ray from camera to point center
    var pVec = pointRay.intersectPlane(camPlane); // point of camPlane & pointRay intersection

        var size = object.size;

        // Distance from point center to mouse ray in pixels.
    var dist = screenXY(inVec, object.cam).distanceTo(screenXY(pVec, object.cam, size));

    if (dist < object.pointSize / 2) {

        // THE ALGORITHM ENDS HERE
        var intersectPoint = ray.closestPointToPoint( point );
        intersectPoint.applyMatrix4( matrixWorld );

        var distance = raycaster.ray.origin.distanceTo( intersectPoint );

        if ( distance < raycaster.near || distance > raycaster.far ) return;

        intersects.push( {

            distance: distance,
            distanceToRay: Math.sqrt( rayPointDistanceSq ),
            point: intersectPoint.clone(),
            index: index,
            face: null,
            object: object

        } );

    }

}

function screenXY:

function screenXY(obj, cam, size = {}){
  if (!obj) return new THREE.Vector3();

  var vector = obj.clone();

  var widthHalf = ((size.width || window.innerWidth)/2);
  var heightHalf = ((size.height || window.innerHeight)/2);

  vector.project(cam);

  vector.x = ( vector.x * widthHalf ) + widthHalf;
  vector.y = - ( vector.y * heightHalf ) + heightHalf;
  vector.z = 0;

  return vector;

};

!!! It raycasts points as circles, not as squares.

The algorithm compares distance from ray projected from camera to point center (dist variable) to half of the pointSize.

points.pointSize = 64;
points.cam = camera; // perspective camera
points.size = renderer.getSize(); // renderer size