luicfrr / react-native-vision-camera-face-detector

Vision Camera Frame Processor Plugin to detect faces using MLKit Face Detector
https://www.npmjs.com/package/react-native-vision-camera-face-detector
MIT License
105 stars 20 forks source link

[BUG 🐛] contours is always empty using contourMode: "all" #97

Closed cursan-a closed 1 week ago

cursan-a commented 1 week ago

Describe the bug:

I tried to get contours of a face using contourMode: "all" in FaceDetectionOptions but I always got an empty object:

 [
  {
    "bounds": {
      "height": 409,
      "width": 409,
      "x": 43,
      "y": 412
    },
    "contours": {},
    "leftEyeOpenProbability": 0.9938164353370667,
    "pitchAngle": 12.616621017456055,
    "rightEyeOpenProbability": 0.9974306225776672,
    "rollAngle": -2.7283248901367188,
    "smilingProbability": 0.0071668741293251514,
    "yawAngle": -10.873421669006348
  }
]

Minimum reproducible example:

It's just a fresh new expo bare workflow project with the following dependencies:

  "dependencies": {
    "@expo/vector-icons": "^14.0.2",
    "@react-navigation/native": "^6.0.2",
    "expo": "~51.0.28",
    "expo-build-properties": "~0.12.5",
    "expo-constants": "~16.0.2",
    "expo-font": "~12.0.9",
    "expo-linking": "~6.3.1",
    "expo-router": "~3.5.23",
    "expo-splash-screen": "~0.27.5",
    "expo-status-bar": "~1.12.1",
    "expo-system-ui": "~3.0.7",
    "expo-web-browser": "~13.0.3",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "react-native": "0.74.5",
    "react-native-gesture-handler": "~2.16.1",
    "react-native-reanimated": "~3.10.1",
    "react-native-safe-area-context": "4.10.5",
    "react-native-screens": "3.31.1",
    "react-native-svg": "15.2.0",
    "react-native-vision-camera": "^4.5.3",
    "react-native-vision-camera-face-detector": "^1.7.1",
    "react-native-web": "~0.19.10",
    "react-native-webview": "13.8.6",
    "react-native-worklets-core": "^1.3.3",
    "zustand": "^5.0.0-rc.2"
  },
  "devDependencies": {
    "@babel/core": "^7.20.0",
    "@types/jest": "^29.5.12",
    "@types/react": "~18.2.45",
    "@types/react-test-renderer": "^18.0.7",
    "jest": "^29.2.1",
    "jest-expo": "~51.0.3",
    "react-test-renderer": "18.2.0",
    "typescript": "~5.3.3"
  },

Expected behavior:

I was hopping to get contours point as in ML Kit documentation

I have tryed: Playing with different parameters landmarkMode none/all, trackingEnabled: true/false, performanceMode: accurate/false, etc. EDIT : Since, I tried to force the GoogleMLKit dependency in the podspec file (5.0 & 4.0) but still have the same issue. I ran my app using Xcode and I add some debug log in VisionCameraFaceDetector.swift and face.contours is indeed empty. Everything looks good in FaceDetectorOptions So it's really weird.

Device:

cursan-a commented 1 week ago

Ok I tested everything and anything with the version of "GoogleMLKit/FaceDetection" and I modified the use of faceDetector from VisionCameraFaceDetector.podspec: no success!

I finally updated the iOS version of my phone to 15.8.3 and miracle it works!

GoogleMLKit/FaceDetection requires the minimum iOS 12 version but I guess that for some strange reason it doesn't work with iOS 14 either 🤷‍♂️