comparison of two videos / inputs to determine alikeness / similarity

[8secz-johndpope]

Did you consider this problem? We want to run challenges , say for a given dance - is the contestant matching the original video.... I looked into it a little with dynamic time warping - but wonder if you have encountered this....

[d1l4y]

It's is a great use that I did't considered, but it looks possible! maybe posekit could help with it.

[8secz-johndpope]

@d1l4y - could I animate the robot.usdz with the captured arframe / posekit data?

clearly - when we're tracking body / all of this is correctly positioned.

but I want to move person out of frame - and have the skeleton / robot move to pre-recorded sequence.

func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {

        if isCapturePlay { return }
        for anchor in anchors {
            guard let bodyAnchor = anchor as? ARBodyAnchor else { continue }

            // Update the position of the character anchor's position.
            let bodyPosition = simd_make_float3(bodyAnchor.transform.columns.3)
            characterAnchor.position = bodyPosition + characterOffset
            // Also copy over the rotation of the body anchor, because the skeleton's pose
            // in the world is relative to the body anchor's rotation.
            characterAnchor.orientation = Transform(matrix: bodyAnchor.transform).rotation

            print(poseKit.BodyTrackingPosition(bodyAnchor: bodyAnchor))
            if let character = character, character.parent == nil {
                // Attach the character to its anchor as soon as
                // 1. the body anchor was detected and
                // 2. the character was loaded.
                characterAnchor.addChild(character)

            }

        }

// my current approach is to stash all the [ARAnchor] frames into an array
// then play them back using a timer.

     func playCapturedFrame(capturedData anchor: ARBodyAnchor) {
        //        for anchor in capturedData {
        // Update the position of the character anchor's position.
        let bodyPosition = simd_make_float3(anchor.transform.columns.3)

        playbackCharacterAnchor.position = bodyPosition + playbackCharacterOffset
        // Also copy over the rotation of the body anchor, because the skeleton's pose
        // in the world is relative to the body anchor's rotation.
        playbackCharacterAnchor.orientation = Transform(matrix: anchor.transform).rotation

        if let playbackCharacter = playbackCharacter, playbackCharacter.parent == nil {
            // Attach the character to its anchor as soon as
            // 1. the body anchor was detected and
            // 2. the character was loaded.
            playbackCharacterAnchor.addChild(playbackCharacter)
        }
    }

but something is off...

https://gist.github.com/8secz-johndpope/1ba5920a6af75936b4e6a534899d1bea

[d1l4y]

Hey, this reminded me of a tweet from dan miller, he accomplished something close to your goal using Unity's ARFoundation, as ARFoundation uses ARKit, this should be possible.

https://twitter.com/danmillerdev/status/1216725207773106177?s=21

I am not sure of what you should do, but I would start gathering the data from ARBodyAnchor and then try to reuse it