hiukim / mind-ar-js

Web Augmented Reality. Image Tracking, Face Tracking. Tensorflow.js
MIT License
2.12k stars 391 forks source link

How to get the changed position and rotation of an a-box or a-plane #514

Open jonathanneels opened 2 months ago

jonathanneels commented 2 months ago

Hi fantastic people,

In my latest project, I am attempting to retrieve the position of either an 'a-box' or an 'a-plane' when I point the camera towards an image. The Three.js figure is visible, but when I use the code document.querySelector("a-box").object3D, I can see all its information, albeit unaltered. Even when I shake my phone, the values of scale, position, and rotation remain as the default information. I have also tried setting document.querySelector("a-box").matrixAutoUpdate = true, but it made no difference.

For some reason, the box (or plane) object does not provide its updated position, despite being visibly altered.

Note, this is my "scene":

  <a-scene mindar-image="imageTargetSrc: static/assets/lady/targets.mind; maxTrack: 2" color-space="sRGB" renderer="colorManagement: true, physicallyCorrectLights" vr-mode-ui="enabled: false" device-orientation-permission-ui="enabled: false">
               <a-assets>
    <img id="card" src="static/assets/softmind/card.png" />
     </a-assets>

      <a-entity mindar-image-target="targetIndex: 0">
                <a-box src="#card" position="0 0.1 0" color='purple' material='opacity: 0.8;' height="0.552" width="1" rotation="0 0 0"></a-box>

      </a-entity>

               <a-camera position="0 0 0" look-controls="enabled: false"></a-camera>

     </a-scene>

Note 2: I know it must be possible, because I already made it work with vanilla AR.js & aframe.

Could someone provide me with a clue?

Thank you in advance.

Kind regards, JN

jonathanneels commented 2 months ago

Hi great people

I found a solution. It is based on the following link: https://codepen.io/evces/pen/vYrwQXW In essence I made a custom interactable Three.js scene, so I have complete control over my objects.

const THREE = window.MINDAR.IMAGE.THREE;
      window["mindarThree"] = new window.MINDAR.IMAGE.MindARThree({
    container: document.querySelector("#container"),
    imageTargetSrc: "static/assets/lady/targets.mind"
      });
      const {renderer, scene, camera} = window["mindarThree"] ;
      const anchor = window["mindarThree"] .addAnchor(0);
      const geometry = new THREE.BoxGeometry(1, 1,1);
      const material = new THREE.MeshBasicMaterial( {color: 0x00ffff, transparent: true, opacity: 0.5} );
        window["plane"] = new THREE.Mesh( geometry, material );
      anchor.group.add( window["plane"]);
       const start = async() => {
    await window["mindarThree"].start();initMarkCheck();
    renderer.setAnimationLoop(() => {
    window["planepos"] =  window["plane"].getWorldPosition(new THREE.Vector3());
        window["planescale"] =  window["plane"].getWorldScale(new THREE.Vector3());

        var quaternion = new THREE.Quaternion();//REF/ https://stackoverflow.com/questions/55192521/what-to-do-now-three-getworldrotation-has-gone
   window["planerot"] =  window["plane"].getWorldQuaternion( quaternion )

    let campos = camera.getWorldPosition(new THREE.Vector3());
    //console.log(`Camera: ${campos.x},${campos.y},${campos.z}`);
    //console.log(`Plane: ${window["planepos"].x},${window["planepos"].y},${window["planepos"].z}`);

    window["plane"].lookAt(new THREE.Vector3());
      renderer.render(scene, camera);
    });
      }

I hope someone will be able to use this information (was a looooong search).

I wish you all a great day.

Kind regards JN