google-ar / arcore-android-sdk

ARCore SDK for Android Studio
https://developers.google.com/ar
Other
4.98k stars 1.22k forks source link

getAllTrackables(AugmentedFace.class) tracks a single face #978

Open shanealv opened 4 years ago

shanealv commented 4 years ago

SPECIFIC ISSUE ENCOUNTERED

Using the following code for getting a list of augmented faced from the camera:

Collection<AugmentedFace> faceList = sceneView.getSession().getAllTrackables(AugmentedFace.class);
Log.i("facetracking", "detected: " + faceList.size());

I am consistently only getting 0 or 1 face detected, as output logs have indicated:

2019-12-30 23:05:48.829 10685-10685/com.google.ar.sceneform.samples.augmentedfaces I/facetracking: detected: 1
2019-12-30 23:05:48.880 10685-10685/com.google.ar.sceneform.samples.augmentedfaces I/facetracking: detected: 1
2019-12-30 23:05:48.945 10685-10685/com.google.ar.sceneform.samples.augmentedfaces I/facetracking: detected: 0
2019-12-30 23:05:49.014 10685-10685/com.google.ar.sceneform.samples.augmentedfaces I/facetracking: detected: 0
2019-12-30 23:05:49.081 10685-10685/com.google.ar.sceneform.samples.augmentedfaces I/facetracking: detected: 1
2019-12-30 23:05:49.130 10685-10685/com.google.ar.sceneform.samples.augmentedfaces I/facetracking: detected: 1

This is despite having two faces in frame.

In fact, if I cover one face, tracking immediately swaps over to the second face, and the initial face is subsequently ignored until the second face is covered.

As per the sample code, I have a section for adding faces as they appear ready for tracking:

for (AugmentedFace face : faceList) {
    if (!faceNodeMap.containsKey(face)) {
    Log.i("facetracking", "found: " + face.toString() + " " + face.getTrackingState());
    AugmentedFaceNode faceNode = new AugmentedFaceNode(face);
    ...
}

It appears that the conditional is only satisfied once, resulting in the following log message appearing once:

2019-12-30 23:04:54.598 10685-10685/com.google.ar.sceneform.samples.augmentedfaces I/facetracking: found: com.google.ar.core.AugmentedFace@3ea1b78 TRACKING

I also have this as the code to stop tracking:

while (iter.hasNext()) {
    Map.Entry<AugmentedFace, AugmentedFaceNode> entry = iter.next();
    AugmentedFace face = entry.getKey();
    if (face.getTrackingState() == TrackingState.STOPPED) {
        Log.i("facetracking", "lost: " + face.toString());
        ...
    }
}

Which doesn't print to the log at all, even when I've clearly put my phone down and I get repeated "detected: 0" log messages.

TL;DR: Getting all trackable faces is really only giving my one, and that face is seeming always in a TRACKING state.

VERSIONS USED

STEPS TO REPRODUCE THE ISSUE

  1. Add the log outputs to the sample augmented faces app
  2. Run the app with multiple faces in frame
  3. Individually cover each face and observe log output (or run a debugger)

WORKAROUNDS (IF ANY)

n/a

ADDITIONAL COMMENTS

Is there perhaps some documentation I've missed marking this as a limitation? The examples and API suggest that multiple faces were meant to be detected, although clearly that isn't the actual behavior in this case.

shanealv commented 4 years ago

Update:

If the stop condition is set to PAUSED instead of STOPPED, it does work. At least, a face will be removed. So the state is changing, just not in a way the example code expects. And of course, we still are only getting one face tracked.