Is @tensorflow-models/face-landmarks-detection@1.0.6 not performed on the react native platform?

[adjh54ir]

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow.js): No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): localhost
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Galaxy S21 5G(SM-G991N) or ALL
• TensorFlow.js installed from (npm or script link): yarn 1.22.22
• TensorFlow.js version (use command below): @tensorflow/tfjs(3.11.0)
• Browser version: Android OS 14
• Tensorflow.js Converter Version: 3.11.0

Describe the current behavior

The following error occurred: TypeError: Cannot read property 'includes' of undefined, js engine: hermes.

Describe the expected behavior

hello!

I have been using version @tensorflow-models/face-landmarks-detection@0.0.3 in the React Native platform environment. I recently performed an update to version 1.0.6, but confirmed that the update did not work and the following error occurred. After checking the error, I determined that it was a method called in a web environment rather than a mobile environment, so I was curious to see if it was actually performed. The error was TypeError: Cannot read property 'includes' of undefined, js engine: hermes and was confirmed to be an error caused by loading the @mediapipe/face_mesh library in the faceLandmarkDetection.SupportedModels.MediaPipeFaceMesh section.

"dependencies": {
    "react": "18.2.0",
    "react-native": "0.72.4",
    "@tensorflow-models/blazeface": "^0.1.0",
    "@tensorflow-models/face-landmarks-detection": "1.0.6",
    "@tensorflow/tfjs": "3.11.0",
    "@tensorflow/tfjs-backend-cpu": "3.11.0",
    "@tensorflow/tfjs-backend-webgl": "3.11.0",
    "@tensorflow/tfjs-converter": "^3.11.0",
    "@tensorflow/tfjs-core": "3.11.0",
    "@tensorflow/tfjs-react-native": "0.8.0",
    "@mediapipe/face_detection": "^0.4.1646425229",
    "@mediapipe/face_mesh": "^0.4.1633559619"
}

Please refer to the source code as below.

import * as faceLandmarkDetection from '@tensorflow-models/face-landmarks-detection';
useEffect(() => {
    loadModel();
}, []);

const loadModel = async () => {
    const model = faceLandmarkDetection.SupportedModels.MediaPipeFaceMesh; // error 
    console.log("model :: ", model)
    const detectorConfig = {
        runtime: 'mediapipe', // or 'tfjs'
        solutionPath: 'base/node_modules/@mediapipe/face_mesh',
    }
    console.log(detectorConfig)
    const detector: faceLandmarkDetection.FaceLandmarksDetector = await faceLandmarkDetection.createDetector(model, detectorConfig);

    console.log(detector)
}

Please reply. Thank you!

Standalone code to reproduce the issue Provide a reproducible test case that is the bare minimum necessary to generate the problem. If possible, please share a link to Colab/CodePen/any notebook.

Other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

[shmishra99]

Hi @adjh54ir ,

I am able to use faceLandmarkDetection.createDetector with the tfjs runtime, but I am getting a TypeError: t.FaceMesh is not a constructor error while using the MediaPipe runtime. I have imported the following dependencies in myApp.js:

import '@mediapipe/face_mesh';
import '@tensorflow/tfjs-core';
// Register WebGL backend.
import '@tensorflow/tfjs-backend-webgl';
import * as faceLandmarksDetection from '@tensorflow-models/face-landmarks-detection';   

Thank you!