🐛 camera freezing when pointing to a "Face"

marcosrjjunior commented 4 months ago

What's happening?

The camera is freezing when pointing to a face. Please check videos below.

Found these issues pointing to the same error: https://github.com/mrousavy/react-native-vision-camera/issues/2313, https://github.com/mrousavy/react-native-vision-camera/pull/2329. Not sure if is direct related because I'm not even using any zoom option.

Seems like the camera is trying to perform some face identification when focus is on a face. I'm guessing it is related to the iOS version.

**

This issue is also happening when using a different camera library.
I could not reproduce on the "ShadowLens 1.1.2 (17)" app.

Reproduceable Code

function App() {
  const device = useCameraDevice('back')

  if (device == null) return <View />

  return (
    <Camera
      style={StyleSheet.absoluteFill}
      device={device}
      isActive
      enableZoomGesture={false}
    />
  )
}

Relevant log output

output on the videos above.

[unknown/unknown: Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" UserInfo={NSLocalizedDescription=Cannot Complete Action, NSLocalizedRecoverySuggestion=Try again later.}]

Camera Device

{
  "isMultiCam": false,
  "minFocusDistance": 10,
  "formats": [],
  "name": "Back Camera",
  "minExposure": -8,
  "hasFlash": false,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "neutralZoom": 1,
  "supportsFocus": true,
  "supportsRawCapture": false,
  "hasTorch": false,
  "maxZoom": 95.625,
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "sensorOrientation": "landscape-left",
  "minZoom": 1,
  "hardwareLevel": "full",
  "position": "back"
}

Device

iPad iOS 17.5.1 9th gen. Couldn't reproduce on 8th gen or iPhone 13 Pro. Maybe something related to the hardware performance.

VisionCamera Version

first identified on "2.16.7", also happening on 4.3.2 after update

Can you reproduce this issue in the VisionCamera Example app?

I didn't try but followed examples and even tried disabling some props.

Additional information

[X] I am using Expo
[ ] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

maintenance-hans[bot] commented 4 months ago

Guten Tag, Hans here.

[!NOTE] New features, bugfixes, updates and other improvements are all handled mostly by @mrousavy in his free time. To support @mrousavy, please consider 💖 sponsoring him on GitHub 💖. Sponsored issues will be prioritized.

arjunmenon1302 commented 4 months ago

bump

mrousavy commented 4 months ago

@arjunmenon1302 why are you bumping an issue on the same exact day it was created at?

mrousavy commented 4 months ago

@marcosrjjunior sorry I cannot help you. I cannot reproduce the issue with the sample code you provided (tested on iPhone 15 Pro and iPhone 8), and if you didn't try to reproduce it in the example app then we don't know where it is coming from. Maybe you are using a Frame Processor Plugin that does something weird.

kurt-idu commented 4 months ago

bump

mrousavy commented 4 months ago

😂

mrousavy commented 4 months ago

@marcosrjjunior AVFoundationErrorDomain code -11819 means that the daemon has crashed, I think you need to file this bug using Apple's Feedback Assistant.

I don't think this is related to VisionCamera at all, but it doesn't hurt to also tell them that you use VisionCamera (provide a link to the github repo)

marcosrjjunior commented 3 months ago

@mrousavy just an update on this issue.

we were able to workaround that by using react-native-camera, with the option autoFocus="off".

The issue was solved just by disabling that autoFocus prop. To improve the experience a lit bit, we are using the autoFocusPointOfInterest with a "slider" to adjust the focus.

I couldn't find a way to force to disable the auto focus on the current vision camera. Thoughts?

-- UPDATE:

After further investigations, disabling these options seems to fix the issue.

self.captureDevice.automaticallyAdjustsFaceDrivenAutoFocusEnabled = NO;
self.captureDevice.faceDrivenAutoFocusEnabled = NO;

is there a way to override somewhere in the camera session without a manual patch?

mrousavy / react-native-vision-camera