Motion capture

[hungphamvn]

Hi, Do you have any example of human motion capture?

[olexale]

Hi @hungphamvn, Currently, the plugin doesn't have this functionality. Though, it is easy to add it. What exactly you would like to see in an example?

[hungphamvn]

Hi @olexale, Appreciate your response, I need it can capture the human body is moving and detect hands or feet are bending or moving (kinda skeleton visualize) so I could calculate value then fire event in realtime. Could you please help me with that?

[olexale]

I can easily add ARBodyTrackingConfiguration for a body tracking and ARBodyAnchor which exposes the skeleton transformation matrix and transformation matrixes of joints (root, head, left hand, etc.).

With that be a solution to your problem?

P.S. I can add a skeleton visualization as well, but it's a little bit different functional.

[hungphamvn]

Exactly what I am looking for... Thanks @olexale .

[olexale]

Done. Please review the latest plugin version from the GitHub. I will push the updated version to pub spec after we test it a bit.

Here is an example of getting a local matrix transformation for one's head:

import 'package:arkit_plugin/arkit_plugin.dart';
import 'package:flutter/material.dart';

class BodyTrackingPage extends StatefulWidget {
  @override
  _BodyTrackingPageState createState() => _BodyTrackingPageState();
}

class _BodyTrackingPageState extends State<BodyTrackingPage> {
  ARKitController arkitController;

  @override
  void dispose() {
    arkitController?.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) => Scaffold(
        appBar: AppBar(title: const Text('Body Tracking Sample')),
        body: Container(
          child: ARKitSceneView(
            configuration: ARKitConfiguration.bodyTracking,
            onARKitViewCreated: onARKitViewCreated,
          ),
        ),
      );

  void onARKitViewCreated(ARKitController arkitController) {
    this.arkitController = arkitController;
    this.arkitController.onAddNodeForAnchor = _handleAddAnchor;
  }

  void _handleAddAnchor(ARKitAnchor anchor) {
    if (!(anchor is ARKitBodyAnchor)) {
      return;
    }
    final ARKitBodyAnchor body = anchor;
    final headMatrix = body.skeleton.localTransforms['head'];
  }
}

[hungphamvn]

You are awesome @olexale . I can't wait to play with it now 😄

[olexale]

Hey @hungphamvn . Did you have time to review this functionality? Any comments regarding the API? Thanks!

[hungphamvn]

Hi @olexale, I'm trying to apply your code into my project but it seems to have some issues, about ARKitConfiguration how can I add bodyTracking into its enum? and _handleAddAnchor function. I just a newbie with flutter so hope you could guide me more a bit with that Thanks a tons ...

[olexale]

Hi! It should be there already. But as I didn't push the source to the pub, you need to depend on this plugin in your pubspec.yml like that:

dependencies:
  arkit_plugin:
    git:
      url: https://github.com/olexale/arkit_flutter_plugin

[hungphamvn]

Yeah. Finally can run the project with your plugin. But nothing appears on the screen. Is that function _handleAddAnchor haven't complete, isn't it @olexale ?

[olexale]

Nothing visual is doing there. The sample gets the local transformation matrix for one's head. Now you can use it for movement detection, or any other stuff. Skeleton visualization requires a little bit more work to do.

[hungphamvn]

Oops! I see. So you could add Skeleton visualization later, right?

[olexale]

Sure, as soon as someone will need it.

[hungphamvn]

Awesome...!!! I think I will need a better iPhone to play with this ( iPhone 6s now cause the error The provided configuration is not supported on this device).

And Many thanks for your great works!