🐛 🐛 [^4.0.0-beta.16] - camera is very slow and lower FPS with useSkiaFrameProcessor and detect faces using react-native-vision-camera-face-detector

[karventhan13]

What's happening?

🐛 🐛 [^4.0.0-beta.16] - camera very slow and lower FPS with useSkiaFrameProcessor when using detect faces usingreact-native-vision-camera-face-detector

Reproduceable Code

const { detectFaces } = useFaceDetector({
    performanceMode: 'accurate',
    contourMode: 'all',
    landmarkMode: 'all',
    classificationMode: 'all',
    trackingEnabled : true
  })

 const handleFacesDetection = Worklets.createRunOnJS((
    result : DetectionResult
  ) => { 
    console.log( 'detection result', result )
})
  const frameProcessor = useSkiaFrameProcessor((frame) => {
    'worklet'

    frame.render() 

    const result = detectFaces({
      frame
    },
    ) 
    handleFacesDetection(result) 

  }, [])

  return (
    <SafeAreaView style={styles.container}>
      {!hasPermission && <Text>No Camera Permission.</Text>}
      {hasPermission && device != null && (
        <Camera
          style={StyleSheet.absoluteFill}
          device={device}
          isActive={true}
          photo={true}
          video={true}
          format={format}
          onError={(err)=> { console.log("err **",err)}}
          fps={fps}
          enableFpsGraph={true}
          frameProcessor={frameProcessor}  
          videoHdr={format.supportsVideoHdr}
          photoHdr={format.supportsPhotoHdr}
          enableZoomGesture={false}
        />
      )}
    </SafeAreaView>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
  },
});

Relevant log output

noe

Camera Device

noe

Device

vivo y21

VisionCamera Version

[^4.0.0-beta.16]

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• [X] I am using Expo
• [X] I have enabled Frame Processors (react-native-worklets-core)
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[anudit]

+1 Bottlenecking on ~15 FPS on Pixel 8

"expo": "~50.0.17",
"react": "18.2.0",
"@shopify/react-native-skia": "^1.2.3",
"react-native": "0.73.7",
"react-native-reanimated": "^3.8.1",
"react-native-vision-camera": "4.0.0-beta.16",
"react-native-vision-camera-face-detector": "^1.5.0",
"react-native-worklets-core": "^1.1.1",

[mrousavy]

No log output, no example video, no camera device, not reproduced in example, nothing. Closing issue due to lack of information.

[aguedob]

Im just suffering the same issue, but I think it's because detectFaces function is running synchronously, right @mrousavy?