needle-tools / needle-engine-support

Needle Engine is a web-based runtime for 3D apps. It runs on your machine for development, and can be deployed anywhere. It is flexible, extensible and includes networking and XR - across platforms. Documentation at https://docs.needle.tools
547 stars 46 forks source link

this.context.physics.raycast() stops hitting targets in AR after the first time it hits an object #113

Closed ROBYER1 closed 1 year ago

ROBYER1 commented 1 year ago

Windows 11, Unity 2022.1.20f1, Exporter 2.47.2-pre I have found that using physics.raycast in AR seems to only hit objects once then never allows hitting them again. Outside of WebXR AR, the same raycast call always hits targets. I've been scratching my head at this all afternoon as it was reproducible in AR on iOS with Mozilla WebXR viewer and also in Chrome Desktop using XR Simulator Extension with Samsung S8 profile

I have:

start()
{
    document.addEventListener("pointerdown", evt => {
        //console.log(evt.target);
        this.touched();
    });
    document.addEventListener("pointerup", evt => {
        this.unTouched();
    });

    document.addEventListener("touchstart", evt => {
        this.touched();
    });
    document.addEventListener("touchend", evt => {
        this.unTouched();
    });
}

public touched()
{
 console.log("touched");
 this.setTargetFromRaycast();
}

public unTouched()
{
 console.log("unTouched");
 this.clearTarget();
}

private setTargetFromRaycast() {
   const rc = this.context.physics.raycast();

    console.log(rc);

    this._raycaster.setFromCamera(rc, this.context.mainCamera);
    for (const hit of rc) {
        console.log(hit.object.name);
        if (hit.distance > 0 && GameObject.isActiveInHierarchy(hit.object)) {
            console.log("Set target", hit.object.name, hit.object);
            this.mainTarget = hit.object as THREE.Object3D;
            this.arctrl = GameObject.getComponentInParent(this.mainTarget, ARControls);
            if(this.arctrl != null)
            {
            if (this.orbit) this.orbit.enabled = false;
            GameObject.setActive(this.arctrl.target, true);
            this.isSelected = true;
            }
            else
            {
                return
            }
            break;
        }
    }
}

In desktop using Pointer events pointerevents

In AR using touch events (using WebXR simulator for logs on desktop) Screengrab: image

Realtime: touchevents

Minimal repro project: ARDemo-221121_uw.zip

ROBYER1 commented 1 year ago

Could be a fluke that pointerdown can fire once in WebXR when it turns out that the touches are not pulling through in WebXR - see related question here https://discord.com/channels/717429793926283276/1044286747577692253

marwie commented 1 year ago

Yes - as said in discord before: touches in AR right now are actually WebXR controllers so you mix some things up in your test because you raycast using a touch event but not give the raycast method a ray.

Instead you should create a ray from your current touch position and use that for raycasting.

ROBYER1 commented 1 year ago

I see, closing as I think this is a non issue and the random pointer down event firing was just catching me off guard when investigating

ROBYER1 commented 1 year ago

I'll share back if I figure out how to get touch events actually firing too. Would be useful to have some example of that in the samples if I do!