How to face tracking with FaceDetector and Bitmap?

[wdgk]

This is a question.

I want to face tracking with FaceDetector like this. https://github.com/googlesamples/android-vision/tree/master/visionSamples/FaceTracker

In this sample, set ByteBuffer to Frame.imageData like this.

 outputFrame = new Frame.Builder()
            .setImageData(mPendingFrameData, mPreviewSize.getWidth(),
                                mPreviewSize.getHeight(), ImageFormat.NV21)
            .setId(mPendingFrameId)
            .setTimestampMillis(mPendingTimeMillis)
            .setRotation(mRotation)
            .build();

But, I set bitmap to Frame like this.

//data is image data as byte.
bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);
outputFrame = new Frame.Builder()
            .setBitmap(bitmap)
            .setId(frameId)
            .setTimestampMillis(time)
            .build();

And use FaceDetector like this.

    FaceDetector fd = new FaceDetector.Builder(this.context)
                        .setTrackingEnabled(true)
                        .setClassificationType(FaceDetector.ALL_CLASSIFICATIONS)
                        .build();

        fd.setProcessor(new MultiProcessor.Builder<Face>(new GraphicFaceTrackerFactory()).build());
        fd.receiveFrame(frame);

     private class GraphicFaceTrackerFactory implements MultiProcessor.Factory<Face> {
        @Override
        public Tracker<Face> create(Face face) {
            return new GraphicFaceTracker();
        }
     }

private class GraphicFaceTracker extends Tracker<Face> {

    GraphicFaceTracker() {}

    @Override
    public void onNewItem(int faceId, Face face) {
        Log.d(Constants.LOG_TAG, "onNewItem. Face Id => "+faceId);//here
    }

    @Override
    public void onUpdate(FaceDetector.Detections<Face> detectionResults, Face face) {
    }

    @Override
    public void onMissing(FaceDetector.Detections<Face> detectionResults) {

    }

    @Override
    public void onDone() {

    }
}

I expected that same faceId will be entered in onNewItem() as long as same face remain in the frame, but entered id was different every time. If I use bitmap instead of the bytebuffer, does it not works properly?

【Additional info】 ・In my application, it takes still pictures continuously at 7.5fps.

[pm0733464]

The faceId does not represent the results of facial recognition, and the API does not support facial recognition (i.e., identifying a specific individual).

The API does support continuous tracking of a face, based upon the position and motion of the face over time. When tracking a face in video, if the motion was relatively steady then the ID will be the same. But if the face is lost and a new face is detected (even if it was the same face as before), it will issue a new faceId to represent the start of a new face tracking session.

[wdgk]

Thank you for your reply.

What I care about is that in the sample application we can track the face, but it is not possible in case of my own application.

I think that the difference between the sample application and my application is whether you are using bytebuffer or using bitmap.

Do you know why it is possible to face tracking on sample application and not possible to face tracking on my application?

[pm0733464]

Not sure. The choice of bytebuffer or bitmap shouldn't affect tracking.

If you're only looking to track one face -- the largest face visible -- you probably want to use LargestFaceFocusingProcessor instead of MultiProcessor. See this example:

https://github.com/googlesamples/android-vision/blob/master/visionSamples/googly-eyes/app/src/main/java/com/google/android/gms/samples/vision/face/googlyeyes/GooglyEyesActivity.java#L302

[wdgk]

umm... I want to track multi face finally but I tried LargestFaceFocusingProcessor instead of MultiProcessor.

FaceDetector fd = new FaceDetector.Builder(this.context)
                .setTrackingEnabled(true)
                .setClassificationType(FaceDetector.ALL_CLASSIFICATIONS)
                .build();

fd.setProcessor(new LargestFaceFocusingProcessor.Builder(fd, new GraphicFaceTracker()).build());

As a result, there was no difference with LargestFaceFocusingProcessor and MultiProcessor.

FaceId coming to onNewItem () is a different Id every time.

[Kostanos]

Hi, did you find the solution?