Face expression probabilities random on chrome android

[davidstrahm]

After loading and detecting face from html video element, the probability distribution is correct for a few frames (happy 0.99, sad 0.0000343) etc.

After a few processed frames, the probabilities jump all over the place (see screenshots below).

The same code works fine when using it in chrome on my desktop machine (Nvidia GTX 1060)

Android 8.0.0 Chrome 73.0.3683.90 Samsung Galaxy S7

const model = FaceApiFaceDetectorModel.TINY_FACE_DETECTOR;
const modelBaseUrl = '/assets/tfmodels';
const minConfidence = 0.5;
const inputSize = 256;
const scoreThreshold = 0.5;
const minFaceSize = 200;

await loadFaceDetectorModel(model, modelBaseUrl);
await faceapi.loadFaceExpressionModel(modelBaseUrl);

const options = getFaceDetectorOptions(model, minConfidence, inputSize, scoreThreshold, minFaceSize);

const result = await faceapi.detectSingleFace(video, options).withFaceExpressions();

Might this be some kind of overflow?

Edit: I tried it with the sample application (examples-browser) by updating the "video face tracking" example. The result is the same: Works on desktop, random values on mobile

[justadudewhohacks]

That's odd, especially since these are not even valid probabilities anymore. Looks like something is going wrong in tf.softmax here.

[davidstrahm]

Maybe this is related: https://github.com/tensorflow/tfjs/issues/1488, although face-api does not do batch predictions, does it?

The probabilities are correct on newer phones. Do you have any plans to upgrade your tfjs dependency? Might be this is already fixed.

[justadudewhohacks]

The code that you posted does not do batch processing. Did you try to play around with enabling/disabling the specific features as @annxingyuan described in this issue? Does that help?

[davidstrahm]

Yes, I tried all 3 suggestions. The predictions are "better" when setting tf.ENV.set('WEBGL_PACK', false) (no more huge numbers), but still there are negative values and values in the range from -100 to 100.

[justadudewhohacks]

Ok. All I can do for now is upgrade tfjs-core to latest, which I am working on now.

[davidstrahm]

Thank you, I'll let you know in this issue if that helped

[justadudewhohacks]

Ok I will have to postpone the upgrading, since tfjs-core 1.1.2 seems to not come with the platform specific check, which seems to be implemented in the current master. This breaks the unit tests, since the browser tests utilize a wrong fetch function.

[davidstrahm]

Let me know if we can help

[davidstrahm]

Update: seems to work now on chrome mobile 74.0.3729.136