search

aframevr / aframe

:a: Web framework for building virtual reality experiences.
https://aframe.io/
MIT License
16.65k stars 3.96k forks source link

Entity re-parenting #2425

Open caseyyee opened 7 years ago

caseyyee commented 7 years ago

It's helpful to be able to entities to different parents. Currently, to do this, you need to:

1) flush entity to DOM (to preserve your entity state). 2) copy the entity. 3) add copy to new parent entity element. 4) remove the existing entity.

var entity = document.querySelector('#myelement');
var newParent = document.querySelector('#mynewparent');
entity.flushToDom();
var copy = entity.cloneNode();
newParent.appendChild(copy);
entity.parentNode.removeChild(entity);

To do this in three.js, it's a simple matter of changing the object3d.parent. DOM parentNode and parentElement are read only properties, so you cannot re-write to a new parent. Not sure if this applies any differently to custom element/web component.

It would be nice to just:

entity.parent = newParent;

Also welcome any suggestions for any alternative way of doing this that I may have missed.

dmarcos commented 7 years ago

@caseyyee what are the use cases for reparenting you have in mind?

ngokevin commented 7 years ago

There are Entity.add and Entity.addToParent, but then they wouldn't be represented in the DOM. Maybe those methods should check if the DOM node is not attached, and attach if so.

Use case I can immediately think of is grabbing an object and putting it in a hand's local coordinate space.

caseyyee commented 7 years ago

Yes, grabbing and object would be a prime example of why you would want to do this.

The add method would work, but I figure you would want to keep the DOM tree faithful to the THREE scene structure.

dmarcos commented 7 years ago

We might want to wrap the methods insertBefore / insertAfter for entities

fernandojsg commented 7 years ago

I guess the quick&dirty way could works but it will recreate all the components from the cloned entity. Ideally the three.js object could be just moved using .parent = blabla but the DOM moved without reinitliazing the whole stuf, but I'm not sure how to do that. I agree that a very common use will be grabbing objects and doing just .parent = blabla it's the way you do typically in unity for example, as it's easy and performant.

ngokevin commented 7 years ago

Trying to re-parent now to grab something. API Options:

Implementation options:

What component data do we want to keep? Do you keep local transforms? Just take original component data? Runtime data?

I might just keep it simpler and do @caseyyee's flushToDOM and copy over. Then up to application to do any processing of data.

ngokevin commented 7 years ago

First, a simpler solution to re-parent. Doesn't take much work so I don't think we need an API. Although we can document it.

entity.flushToDom();
newParent.appendChild(entity);

Second, @dmarcos came up with a much better idea for temporary grabbing. Move the three.js Object3D as a child of the hand's Object3D, and leave the DOM as is. When you ungrab, you can resync the DOM if wanted. I'm gonna try that out.

ngokevin commented 7 years ago

Example code for grabbing / ungrabbing by re-parenting the Object3D:

Grab:

 // World to local transform so position and rotation do not change when moved.
handEl.object3D.updateMatrixWorld();
worldToLocal = new THREE.Matrix4().getInverse(handEl.object3D.matrixWorld);
object3D.applyMatrix(worldToLocal);

Ungrab:

// Get world transforms.
position = activePrimitiveObject3D.getWorldPosition();
rotation = activePrimitiveObject3D.getWorldRotation();

// Move object back to original parent.
sceneEl.object3D.add(object3D);

// Sync world transforms back to entity.
el.setAttribute('position', position);
el.setAttribute('rotation', {
  x: THREE.Math.radToDeg(rotation.x),
  y: THREE.Math.radToDeg(rotation.y),
  z: THREE.Math.radToDeg(rotation.z)
});
machenmusik commented 7 years ago

isn't flushToDOM() expensive?

machenmusik commented 7 years ago

you should be able to move something without owning it, personally i think reparenting should be reserved for more durable reasons. i think of it more like tractor beam unless you are really grabbing to wield, not grab-and-drop. should be able to do tractor beam with 3DOF, if you are moving like in editor not purely physical analog, you want to be able to push and pull much farther than 6DOF range of motion, so trackpad/thumbstick anyway

machenmusik commented 7 years ago

resyncing by using unmodified world position / rotation would only work if no new parent has nonzero offsets, right?

caseyyee commented 7 years ago

I agree with @machenmusik.

There are many general cases where you want the change reflected into the DOM and not just changed on Object3D.

Maybe a more relatable example:

<a-bag></a-bag>
<a-car></a-car>

Now put the bag into the car and drive it around:

<a-car>
  <a-bag></a-bag>
</a-car>

So to answer @ngokevin earlier questions:

A high effort, but maybe worthwhile idea would be to separate the DOM tree from three.js scene graph. Doing this, would make reparenting non-destructive and give us flexibility and performance in other areas as well. Anyways, more pie in the sky than anything else.

ngokevin commented 7 years ago

Yeah, a nice to have. But working around or doing it manually is fairly easy. If we wanted to add DOM syncing or account for controller offsets, we could at the application-level.

And yeah, an A-Frame implementation should separate DOM from three.js somehow.

ghost commented 7 years ago

We have an a-entity with a number of children that we are trying to re-parent to the left or right Vive controller based on the menu pressed. When doing so we receive an TypeError from index.js:270 that states, "position is undefined"

We have tried the flushToDOM() call on the entity prior to the appendChild() that you recommend above. We have also tried the initial list of stuff above where we cloneNode() prior to appendChild() followed by removeChild(). Nothing has worked.

Here's the call stack if it helps, TypeError: position is undefined[Learn More] index.js:270 setPositions/< index.js:270 forEach self-hosted:281:13 setPositions index.js:267 remove index.js:101 value a-entity.js:428:6 [161]</module.exports/</< bind.js:12:13 forEach self-hosted:281:13 value a-entity.js:106:6 m document-register-element.js:2:1464 l document-register-element.js:2:80 o/< document-register-element.js:2:210 r document-register-element.js:2:4796 [16]</</r[a]/O</< document-register-element.js:2:4917

roskelld commented 6 years ago

I've been working on a setup for re-parenting objects, including the camera. I want to do this to utilise the transform updates of the parent, but maintaining the worldspace of the child object as it is parented and unparented.

Use cases: an elevator or tray of objects.

Keeping the worldspace transform and rotation of the child would allow a camera to step on an elevator, move freely whilst on it, then step off on a different floor seamlessly.

I found the following code works great for non-kinematic type objects. 

          child.object3D.parent.updateMatrixWorld();
          // remove child from scene and add it to parent
          THREE.SceneUtils.attach( child.object3D, child.sceneEl.object3D, parent.object3D );

When I test the code out with the camera, however, there's a transform jump as I transition on or off the parent object. It looks like the camera worldtransform is stuck at the initial spawn point. 

CAMERA
[BEFORE]
Child Position (world) Vector3 { x: 1.11, y: 1.6, z: -17.91 }
Old Parent Position Vector3 { x: 0, y: 0, z: 0 }
[AFTER]
New Parent Position Vector3 { x: 0, y: -0.14, z: -20 }
Child Position (world) Vector3 { x: 1.11, y: 1.45, z: -37.94 }
Child Position (local) Vector3 { x: 1.11, y: 1.6, z: -17.94 }

STATIC OBJECT
[BEFORE]
Child Position (world) Vector3 { x: 0, y: 5, z: -12 }
Old Parent Position Vector3 { x: 0, y: 0, z: 0 }
[AFTER]
New Parent Position Vector3 { x: 15, y: 0, z: -3 }
Child Position (world) Vector3 { x: 0, y: 5, z: -12 }
Child Pposition (local) Vector3 { x: -15, y: 5, z: -9 }

Any thoughts on how to resolve this?

ngokevin commented 6 years ago

For reference, the current mode of simulating "grabbing" or re-parenting: https://www.youtube.com/watch?v=Gd-Qc9TuHZ0

Umm off top of my head, I think it's like: have a dummy object under the parent. Invert the entity's world matrix/transforms. Apply dummy object's matrixWorld transforms to entity. Or something similar. If I get around to this again, I'll post code.

PlumCantaloupe commented 6 years ago

What about some sort of temporary parent-constraint for when we don't want to mess with the scene hierarchy, inspired by Maya?

Here is an example component on Glitch. I haven't battle tested it yet but perhaps a start?

Probably should have some keepOffset functionality ...

AFRAME.registerComponent('parent-constraint', {
    //dependencies: ['mdmu-gltf', 'mdmu-controls'],
    schema: {
        parent:                 {type: "selector", default:null},
        position:               {type: "boolean",  default:true},
        rotation:               {type: "boolean",  default:true},
        scale:                  {type: "boolean",  default:true},
        copyParentTransforms:   {type: "boolean",  default:false}, //just want it to match all transforms of "parent" (overrides everything else)
        maintainOffset:         {type: "boolean",  default:false}
    },
    multiple: false, //do not allow multiple instances of this component on this entity
    init: function() {
        this.psuedoParent = null;
    },
    update: function(oldData)  {
        const Context_AF    = this;
        const data = this.data;

        if (Object.keys(data).length === 0) { return; } // No need to update. as nothing here yet

        //model change
        if ( (oldData.parent !== data.parent) && data.parent !== null ) {

            //have to keep checking if everything is ready first ...
            let loopCounter = 0;
            const checkChildParentLoadStatus = () => {
                Context_AF.psuedoParent = document.querySelector('#' + data.parent.getAttribute('id'));
                if (Context_AF.psuedoParent !== null) {
                    if ( Context_AF.psuedoParent.hasLoaded && Context_AF.el.hasLoaded ) {
                        Context_AF.setupConstraint();
                        clearInterval(constraintLoop);
                    }
                }

                if (++loopCounter > 20) {
                    console.log( "Warning! : problems setting parentConstraint" );
                    clearInterval(constraintLoop);
                }
            };
            const constraintLoop = setInterval(checkChildParentLoadStatus, 100);

            // document.querySelector('a-scene').addEventListener('loaded', () => {
            //     Context_AF.setupConstraint();
            // });
        }

        //remove this component if you want to break constraint .... setting to null will do nothing. 
        //To disable temporary set position, rotation, scale to false ..
    },
    setupConstraint: function () {
        const Context_AF    = this;
        const data = this.data;

        Context_AF.originalWorldTransform         = Context_AF.el.object3D.matrixWorld.clone();
        Context_AF.originalLocalTransform         = Context_AF.el.object3D.matrix.clone();
        //Context_AF.originalLocalTransform.compose(Context_AF.el.object3D.position, Context_AF.el.object3D.quaternion, Context_AF.el.object3D.scale); //save for later

        console.log( this.originalWorldTransform );

        let position_P              = new THREE.Vector3();
        let position_C              = new THREE.Vector3();
        let rotation_P              = new THREE.Quaternion();
        let rotation_C              = new THREE.Quaternion();
        let scale_P                 = new THREE.Vector3();
        let scale_C                 = new THREE.Vector3();

        Context_AF.psuedoParent.object3D.matrixWorld.decompose(position_P, rotation_P, scale_P);
        Context_AF.el.object3D.matrixWorld.decompose(position_C, rotation_C, scale_C); //apply saved local transform

        //save transform offset between pseudo-parent and this object
        Context_AF.diffPos = position_C.clone();
        Context_AF.diffPos.sub(position_P);

        //get rot diff. QTransition = QFinal * QInitial^{-1} 
        //https://stackoverflow.com/questions/1755631/difference-between-two-quaternions
        Context_AF.diffQuat = rotation_C.clone();
        rotation_P.inverse();
        Context_AF.diffQuat.multiply(rotation_P);

        Context_AF.diffScale = scale_C.clone();
        Context_AF.diffScale.sub(scale_P);
        Context_AF.diffScale.x = Math.abs(Context_AF.diffScale.x);
        Context_AF.diffScale.y = Math.abs(Context_AF.diffScale.y);
        Context_AF.diffScale.z = Math.abs(Context_AF.diffScale.z);

        //Context_AF.el.object3D.matrixAutoUpdate   = false; //we want to manually update here and not have conflicts elsewhere
    },
    tick: function(time, timeDelta) {
        if (this.psuedoParent !== null) {
            const Context_AF    = this;
            const data          = Context_AF.data;

            let parentObject3D          = Context_AF.psuedoParent.object3D;
            let thisObject3D            = Context_AF.el.object3D;
            let worldMat_Constraint     = new THREE.Matrix4();
            let worldMat_NoConstraint   = new THREE.Matrix4();
            let position_P              = new THREE.Vector3();
            let position_C              = new THREE.Vector3();
            let rotation_P              = new THREE.Quaternion();
            let rotation_C              = new THREE.Quaternion();
            let scale_P                 = new THREE.Vector3();
            let scale_C                 = new THREE.Vector3();

            //get world matrix of pseudo-parent we want to constrain to
            worldMat_Constraint.copy( parentObject3D.matrixWorld ); 

            //get world matrix of this object if we didn't apply a constraint (taking into account local transform)
            worldMat_NoConstraint.copy( thisObject3D.parent.matrixWorld );         
            worldMat_NoConstraint.premultiply( this.originalLocalTransform );

            //const originalWorldTransform = worldMat_NoConstraint.clone();

            //break down into individual transforms ... thanks for the handy function THREEjs!
            worldMat_Constraint.decompose(position_P, rotation_P, scale_P);
            worldMat_NoConstraint.decompose(position_C, rotation_C, scale_C); //apply saved local transform

            //if we want to ignore constrain on one of these transforms we will "reset" it back to what it would be with no constraint applied
            if (!data.position) {
                position_P.copy(position_C);
            }

            if (!data.rotation) {
                rotation_P.copy(rotation_C);
            }

            if (!data.scale) {
                scale_P.copy(scale_C);
            } 

            //if we want to main offset else we don't
            if (data.copyParentTransforms) {
                //recompose world matrix with adjusted transforms
                worldMat_Constraint.compose(position_P, rotation_P, scale_P); 
            }
            else {
                let posMat = new THREE.Matrix4();
                let posMat_Off = new THREE.Matrix4();
                posMat.makeTranslation(position_P.x, position_P.y, position_P.z );
                posMat_Off.makeTranslation(this.diffPos.x, this.diffPos.y, this.diffPos.z );

                let rotMat = new THREE.Matrix4();
                let rotMat_Off = new THREE.Matrix4();
                rotMat.makeRotationFromQuaternion(rotation_P);
                rotMat_Off.makeRotationFromQuaternion(this.diffQuat);

                let scaleMat = new THREE.Matrix4();
                let scaleMat_Off = new THREE.Matrix4();
                if ( scale_P.length() > Number.EPSILON ) { //zero-vector will throw a bunch of errors here ...
                    scaleMat.makeScale(scale_P.x, scale_P.y, scale_P.z);
                }
                if ( this.diffScale.length() > Number.EPSILON ) {
                    scaleMat_Off.makeScale(this.diffScale.x, this.diffScale.y, this.diffScale.z);
                }

                worldMat_Constraint.identity();

                if ( data.maintainOffset ) {
                    if (data.rotation) { 
                        worldMat_Constraint.premultiply( rotMat_Off );
                    }
                    if (data.scale) {
                        worldMat_Constraint.premultiply( scaleMat_Off );
                    }
                    if (data.position) {
                        worldMat_Constraint.premultiply( posMat_Off );
                    }

                    if (data.rotation) { 
                        worldMat_Constraint.premultiply( rotMat );
                    }
                    if (data.scale) {
                        worldMat_Constraint.premultiply( scaleMat );
                    }
                    if (data.position) {
                        worldMat_Constraint.premultiply( posMat );
                    }
                }
                else {
                    if (data.scale) {
                        worldMat_Constraint.premultiply( scaleMat );
                        worldMat_Constraint.premultiply( scaleMat_Off );
                    }
                    if (data.rotation) { 
                        worldMat_Constraint.premultiply( rotMat );
                        worldMat_Constraint.premultiply( rotMat_Off );
                    }
                    if (data.position) {
                        worldMat_Constraint.premultiply( posMat );
                        worldMat_Constraint.premultiply( posMat_Off );
                    }
                }
            }

            //set new matrix and manually update
            const invOriginal = new THREE.Matrix4().getInverse(thisObject3D.matrix);
            thisObject3D.applyMatrix( invOriginal ); //reset this objects matrices
            thisObject3D.applyMatrix( worldMat_Constraint );
        }
    },
    remove: function() {
        //this.el.object3D.matrixAutoUpdate = true;   //give back control of update to framework
    }
});

EDIT: original code was very broken, though this still can be optimized (especially num of vars) EDIT2: fixed a couple of bugs EDIT3: was subverting THREEjs transforms updating pattern. Updated to follow in line with use expectations.

damned commented 4 years ago

bump with reasoning:

this feels to me a disproportionately troublesome and tricky problem to code around, especially for new aframe or 3d developers (like me), for a need that seems likely very common.

workarounds to vary the three object graph away from the a-frame entity graph, or use tracking mechanisms or manually deal with scale and rotations individually are great to get us through, but are:

use case seems pretty clear:

many of the chunks of code we have poking under the covers just no longer work - in part or in whole - so, having versioned code maintained as part of aframe is imho pretty much a necessity

that said, to add to the soon-to-be-out-of-date snippets, here's my contribution :)

it gives reparenting for single entities while maintaining worldspace presence (including scale) which seems to work for my use cases aframe 1.0.4 oculus quest v18

let matrixCopy = (matrix) => new THREE.Matrix4().copy(matrix)
const identityMatrix = new THREE.Matrix4().identity()

function reparent(entity, newParent, done) {
  let intendedWorldMatrix = matrixCopy(entity.object3D.matrixWorld) // this encodes starting pos, rot and scale of entity being moved
  entity.parentElement.removeChild(entity)

  let reparented = entity.cloneNode()
  newParent.appendChild(reparented)

  reparented.addEventListener('loaded', () => {
    // based on parent-world.local matrix multiplication order for child's world matrix:
    //   https://github.com/mrdoob/three.js/blob/dev/src/core/Object3D.js#L560
    //
    // determine required child local matrix thus:
    //   https://math.stackexchange.com/questions/949341/how-to-find-matrix-b-given-matrix-ab-and-a          
    let recalculatedLocalMatrix = new THREE.Matrix4().getInverse(newParent.object3D.matrixWorld).multiply(intendedWorldMatrix)

    let matrixAutoUpdate = reparented.object3D.matrixAutoUpdate
    reparented.object3D.matrixAutoUpdate = false // so that local matrix doesn't get trashed on tick while updating 
    reparented.object3D.matrix.copy(recalculatedLocalMatrix)
    reparented.object3D.applyMatrix(identityMatrix) // to set pos, rot, scale from matrix
    reparented.object3D.matrixAutoUpdate = matrixAutoUpdate
    newParent.object3D.updateMatrixWorld() // ensure all world matrices around moved entity are consistent

    done()
  })
  return reparented
}

FYI aframe 1.0.4 uses three.js r111, so applyMatrix is used, instead of its replacement (applyMatrix4) at three.js head

NB reparenting entities with children of their own would require a little more work - clone recursively (cloneNode(true)) and deal with resetting their matrices as intended (although simpler as only copy from previous local matrices required - i.e. no back-calculation - followed by applyMatrix(identity) on each before updateMatrixWorld() called)

jgbarah commented 3 years ago

I've explored a different solution, based on using object3D.attach to get the coordinates with respect to the intended parent, and then cloning the entity (and later removing the "old" one). See a component implemented following that approach in this answer to "AFrame: reparenting an element keeping its world position, rotation".