search

mrousavy / react-native-vision-camera

📸 A powerful, high-performance React Native Camera library.
https://react-native-vision-camera.com
MIT License
6.64k stars 983 forks source link

🐛 Camera FPS throttles on low light, and torch doesn't always work #2838

Closed wuguishifu closed 2 weeks ago

wuguishifu commented 2 weeks ago

What's happening?

I am using a frame processor on a Pixel 6a/Note 9. I have simplified the frame processor to just not actually perform any calculations. The camera normally runs at 30 FPS according to the graph, but when I cover the camera (with a cloth, my finger, putting it on a table, etc.) the FPS drops to 15. I have tried to disable as much auto stuff as possible on the camera.

Another issue is that the torch does not turn on, even when specified "on" in the camera's props. This happens regardless of the other props. If I edit the file and hot update my app while it's running (in dev mode) it turns on the flashlight. However, if I close and reopen the app, the flashlight once again does not turn on.

These systems worked in v3.9.2 on Android and are currently working in v4 on iOS (iPhone 15pm, iOS 17.x and iPhone 11, iOS 17.x), so I believe these issues are exclusive to Android and v4.

Reproduceable Code

const device = useCameraDevice('back');

const frameProcessor = useFrameProcessor(() => {
  'worklet'
}, []);

const format = useMemo(() => {
  return getCameraFormat(device, [{
    fps: 30
  }]);
}, [device]);

return (
  <Camera
    isActive
    style={StyleSheet.absoluteFill}
    device={device}
    torch='on'
    frameProcessor={frameProcessor}
    enableFpsGraph
    pixelFormat='yuv'
    fps={30}
    format={format}
  />
);

Relevant log output

N/A

Camera Device

{
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "full",
  "maxZoom": 8,
  "minZoom": 1,
  "maxExposure": 20,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 10,
  "minExposure": -20,
  "name": "0 (BACK) androidx.camera.camera2",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}

Device

Pixel 6A (Android 14), Note 9 (Android 10)

VisionCamera Version

4.0.1

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

mrousavy commented 2 weeks ago

The camera normally runs at 30 FPS according to the graph, but when I cover the camera (with a cloth, my finger, putting it on a table, etc.) the FPS drops to 15

This is intended behaviour on Android, devices automatically throttle their FPS to allow more light to reach the sensor.

Also please leave the issue title as prefered in the template.

mrousavy commented 2 weeks ago

Relevant log output

N/A

This is literally the only thing I would've needed to investigate the torch issue. Closing because you left out the required information.

wuguishifu commented 2 weeks ago

Thanks, appreciate the response. I wasn't aware that that was intended Android behavior, thanks for letting me know.

If I'm able to get a log output later on can I reopen this thread or submit a new bug report?

mrousavy commented 2 weeks ago

Yes you can reopen once you have logs

wuguishifu commented 2 weeks ago

Sorry, one last thing. Is the throttling intended behavior even when the lowLightBoost prop is set to false? It was my understanding that the throttling would only happen when that was set to true.

Also, I just wanted to say that this is one of the best react-native libraries IMO and I'm sure everyone appreciates the amount of work you put into maintaining and constantly updating and fixing things!

mrousavy commented 2 weeks ago

Is the throttling intended behavior even when the lowLightBoost prop is set to false? It was my understanding that the throttling would only happen when that was set to true.

Yea it was usually only enabled when lowLightBoost is enabled, but since I migrated from my custom Camera2 implementation to CameraX, this now is no longer configurable. So I guess we just have to accept it, maybe I'll create a feature request in the Google issue tracker but they only have setTargetFps(...), no min or max.

mrousavy commented 2 weeks ago

Also, I just wanted to say that this is one of the best react-native libraries IMO and I'm sure everyone appreciates the amount of work you put into maintaining and constantly updating and fixing things!

Thank you so much, much appreciated!

Hope you don't take offense in me closing issues without logs, but otherwise this just gets way too messy :)

mrousavy commented 2 weeks ago

Wait actually I just double-checked - nevermind, it should actually only throttle the FPS if low-light boost is true, if it is false it should still stick to the FPS... Then this is a bug! https://github.com/mrousavy/react-native-vision-camera/blob/f8b40fd7c7df33d9436ce62cdb2eaa01a652c238/package/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt#L203-L210