search

mrousavy / react-native-vision-camera

📸 A powerful, high-performance React Native Camera library.
https://react-native-vision-camera.com
MIT License
7.51k stars 1.09k forks source link

🐛 Camera start and stop issues #232

Closed G0retZ closed 3 years ago

G0retZ commented 3 years ago

Looks like camera device is struggled to start and stop immediately

What

We are building an app with React Navigation. So each screen is actually a view, that is manipulated by the library on navigation.

I have found that if I "open a screen" with the CameraView being active by default, there are problems:

If I add a delay before making CameraView active like so:

  useEffect(() => {
    if (!visible) {
      setTrueVisible(false);
      return () => {};
    }
    const timeout = setTimeout(() => setTrueVisible(true), 20);
    return () => clearTimeout(timeout);
  }, [visible, setTrueVisible]);

And the same applies to the CameraView destroying. When I navigate back, the camera stays active (can be seen by green dot indicator in iPhone).

But if I first make it inactive and then navigate back after a small delay - all is fine.

Tried to update the library to the lates version (2.4.1) - same result.

Is this delay mandatory or smith else is the source of the issue?

Logs

The problem was found by lack of logs. Actually there are no errors in the logs. Just actual misbehavior... ¯_(ツ)_/¯

Reproducable sample

Try to add this view and alter visible prop. Mb by button.

import React, { useEffect, useMemo, useRef } from 'react';
import { StyleSheet, NativeModules, NativeEventEmitter } from 'react-native';
import {
  Camera,
  useCameraDevices,
  useFrameProcessor,
} from 'react-native-vision-camera';

import useAppState from '../../../hooks/useAppState';
import { useCameraPermissions } from '../hooks';
import { PermissionStatus } from '../types';
import scanQRCodes from '../../../util/vision-camera-qr-scanner';

export default ({ visible, flash, onScannedCodes }: Props) => {
  const camera = useRef<Camera>(null);
  const { status } = useCameraPermissions();
  const devices = useCameraDevices();
  const appState = useAppState();
  const device = useMemo(
    () => (status === PermissionStatus.allowed ? devices.back : undefined),
    [devices, status]
  );
  const isForeground = useMemo(() => visible && appState === 'active', [
    visible,
    appState,
  ]);

  useEffect(() => {
    const subscription = eventEmitter?.addListener(
      'onQRCodesScanned',
      (result) => onScannedCodes?.(result.codes)
    );
    return () => subscription?.remove();
  }, []);

  const frameProcessor = useFrameProcessor(
    (frame) => {
      'worklet';

      scanQRCodes(frame);
    },
    [scanQRCodes]
  );

  return device ? (
    <Camera
      ref={camera}
      style={StyleSheet.absoluteFill}
      device={device}
      torch={visible && flash ? 'on' : 'off'}
      isActive={isForeground}
      frameProcessor={frameProcessor}
      frameProcessorFps={2}
      photo
      video
    />
  ) : null;
};

Environment

G0retZ commented 3 years ago

A released question I asked earlier #190

mrousavy commented 3 years ago
  1. This happens on both iOS and Android?
  2. Can you log isForeground? Does it actually have true when you expect the camera to open?
  3. Can you run the project in Xcode/Android Studio and share the native logs with me? Maybe there's something being logged there...
  4. Does it work if you use the following code: 
    return device ? (
<Camera
  ref={camera}
  style={StyleSheet.absoluteFill}
  device={device}
  isActive={true}
  frameProcessor={frameProcessor}
  frameProcessorFps={2}
  video={true}
/>
) : null;

    ?

  5. Does this always happen or only on fast refresh/hot reload?
G0retZ commented 3 years ago
  1. On Android looks like it works (no frame processor, just takeSnapshots
  2. Yes, I logged, it is true.
  3. Logs are coming (the case without delay)
  4. Same result unfortunately
  5. Happens always despite of hot reload or cold start
G0retZ commented 3 years ago

The iOS log right after navigating to the screen with the camera:

2021-06-28 19:06:29.182929+0300 emma dev[2144:890642] [javascript] 'currentlyFocusedField is deprecated and will be removed in a future release. Use currentlyFocusedInput', '\nTransitioner@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:136882:36\nStackView@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:133740:36\nNavigator@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:133039:38\nKeyboardAwareNavigator@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:133153:38\nNavigationContainer@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:130856:38\nRCTView\nView\nErrorBoundaries@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:463503:36\nNavigator@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:461476:36\nConnect(Navigator)@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:120098:43\nRCTView\nView\nApp@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:124366:36\nConnect(App)@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:120098:43\nRNCSafeAreaProvider\nSafeAreaProvider@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:123005:24\nProvider@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:119675:21\nRoot@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:119477:36\nRCTView\nView\nRCTView\nView\nAppContainer@http://192.168.0.11:8081/index.bundle?platform=ios&dev=true&minify=false&modulesOnly=false&runModule=true&app=com.emma-app.dev:69137:36'
2021-06-28 19:06:29.344607+0300 emma dev[2144:890471] [native] VisionCamera.didSetProps(_:): Updating 13 prop(s)...
2021-06-28 19:06:29.344777+0300 emma dev[2144:891916] [native] VisionCamera.configureCaptureSession(): Configuring Session...
2021-06-28 19:06:29.344805+0300 emma dev[2144:891916] [native] VisionCamera.configureCaptureSession(): Initializing Camera with device com.apple.avfoundation.avcapturedevice.built-in_video:6...
2021-06-28 19:06:29.344843+0300 emma dev[2144:891916] [native] VisionCamera.configureCaptureSession(): Adding Video input...
2021-06-28 19:06:29.349154+0300 emma dev[2144:891916] [native] VisionCamera.configureCaptureSession(): Adding Photo output...
2021-06-28 19:06:29.351966+0300 emma dev[2144:891916] [native] VisionCamera.configureCaptureSession(): Adding Video Data output...
2021-06-28 19:06:29.352410+0300 emma dev[2144:891916] [native] VisionCamera.invokeOnInitialized(): Camera initialized!
2021-06-28 19:06:29.352472+0300 emma dev[2144:891916] [native] VisionCamera.configureCaptureSession(): Session successfully configured!
2021-06-28 19:06:29.352950+0300 emma dev[2144:891916] [native] VisionCamera.configureFormat(): Configuring Format...
2021-06-28 19:06:29.352956+0300 emma dev[2144:891916] [native] VisionCamera.configureDevice(): Configuring Device...
2021-06-28 19:06:29.352986+0300 emma dev[2144:891916] [native] VisionCamera.configureDevice(): Device successfully configured!
2021-06-28 19:06:29.353137+0300 emma dev[2144:891916] [native] VisionCamera.didSetProps(_:): Starting Session...
2021-06-28 19:06:29.975627+0300 emma dev[2144:891916] [native] VisionCamera.didSetProps(_:): Started Session!
G0retZ commented 3 years ago

Looks like it just skipped the frame processor adding?

G0retZ commented 3 years ago

This is Android logs for the same case:

2021-06-28 19:09:09.627 929-929/? W/adbd: timeout expired while flushing socket, closing
2021-06-28 19:09:10.069 17592-20176/com.emmaprod E/ReactNativeJS: 'currentlyFocusedField is deprecated and will be removed in a future release. Use currentlyFocusedInput', '\nTransitioner@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:135346:36\nStackView@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:132204:36\nNavigator@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:131503:38\nKeyboardAwareNavigator@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:131617:38\nNavigationContainer@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:129320:38\nRCTView\nView\nErrorBoundaries@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:460506:36\nNavigator@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:458504:36\nConnect(Navigator)@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:118562:43\nRCTView\nView\nApp@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:122830:36\nConnect(App)@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:118562:43\nRNCSafeAreaProvider\nSafeAreaProvider@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:121469:24\nProvider@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:118139:21\nRoot@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:117941:36\nRCTView\nView\nRCTView\nView\nAppContainer@http://localhost:8081/index.android.bundle?platform=android&dev=true&minify=false&app=com.emmaprod&modulesOnly=false&runModule=true:68170:36'
2021-06-28 19:09:10.111 17592-17592/com.emmaprod E/libc: Access denied finding property "vendor.camera.aux.packagelist"
2021-06-28 19:09:10.104 17592-17592/com.emmaprod W/com.emmaprod: type=1400 audit(0.0:21979): avc: denied { read } for name="u:object_r:persist_camera_prop:s0" dev="tmpfs" ino=2223 scontext=u:r:untrusted_app:s0:c232,c256,c512,c768 tcontext=u:object_r:persist_camera_prop:s0 tclass=file permissive=0
2021-06-28 19:09:10.344 17592-17592/com.emmaprod W/CameraView: CameraViewModule::getAvailableCameraDevices took: 240 ms
2021-06-28 19:09:11.054 17592-17592/com.emmaprod W/CameraX-core_ca: type=1400 audit(0.0:21980): avc: denied { read } for name="u:object_r:persist_camera_prop:s0" dev="tmpfs" ino=2223 scontext=u:r:untrusted_app:s0:c232,c256,c512,c768 tcontext=u:object_r:persist_camera_prop:s0 tclass=file permissive=0
2021-06-28 19:09:11.068 17592-20629/com.emmaprod E/libc: Access denied finding property "persist.vendor.camera.privapp.list"
2021-06-28 19:09:11.070 953-24003/? E/CameraService: tom han set sys.camera 000 ....
2021-06-28 19:09:11.114 17592-17592/com.emmaprod W/Binder:17592_7: type=1400 audit(0.0:21981): avc: denied { read } for name="u:object_r:persist_camera_prop:s0" dev="tmpfs" ino=2223 scontext=u:r:untrusted_app:s0:c232,c256,c512,c768 tcontext=u:object_r:persist_camera_prop:s0 tclass=file permissive=0
2021-06-28 19:09:11.120 17592-24079/com.emmaprod E/libc: Access denied finding property "vendor.camera.aux.packagelist"
2021-06-28 19:09:11.121 17592-24079/com.emmaprod E/libc: Access denied finding property "vendor.camera.aux.packagelist"
2021-06-28 19:09:11.114 17592-17592/com.emmaprod W/Binder:17592_7: type=1400 audit(0.0:21982): avc: denied { read } for name="u:object_r:persist_camera_prop:s0" dev="tmpfs" ino=2223 scontext=u:r:untrusted_app:s0:c232,c256,c512,c768 tcontext=u:object_r:persist_camera_prop:s0 tclass=file permissive=0
2021-06-28 19:09:11.511 17592-20075/com.emmaprod W/OpenGLRenderer: Surface doesn't have any previously queued frames, nothing to readback from
2021-06-28 19:09:11.527 17592-17592/com.emmaprod E/SurfaceViewImpl: PreviewView.SurfaceViewImplementation.getBitmap() failed with error 3
2021-06-28 19:09:11.884 753-753/? W/provider@2.4-se: type=1400 audit(0.0:21983): avc: denied { read } for name="u:object_r:default_prop:s0" dev="tmpfs" ino=2152 scontext=u:r:hal_camera_default:s0 tcontext=u:object_r:default_prop:s0 tclass=file permissive=0
2021-06-28 19:09:11.890 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.894 753-753/? E/walog/SCamera: module name: defualt -> HMD_M582
2021-06-28 19:09:11.895 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.895 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.884 753-753/? W/provider@2.4-se: type=1400 audit(0.0:21984): avc: denied { read } for name="u:object_r:default_prop:s0" dev="tmpfs" ino=2152 scontext=u:r:hal_camera_default:s0 tcontext=u:object_r:default_prop:s0 tclass=file permissive=0
2021-06-28 19:09:11.904 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.904 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.904 753-753/? E/walog/SCamera: module name: defualt -> HMD_M582
2021-06-28 19:09:11.904 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.904 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.904 753-753/? E/walog/SCamera: module name: defualt -> HMD_M582
2021-06-28 19:09:11.904 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.905 753-753/? E/libc: Access denied finding property "persist.wa.log.level"
2021-06-28 19:09:11.908 779-855/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [2, 1]
2021-06-28 19:09:11.908 779-855/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [2, 0]
2021-06-28 19:09:11.908 779-855/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [2, 1]
2021-06-28 19:09:11.908 779-855/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [2, 0]
2021-06-28 19:09:11.908 779-855/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [2, 1]
2021-06-28 19:09:11.908 779-855/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [2, 0]
2021-06-28 19:09:11.908 779-855/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [2, 1]
2021-06-28 19:09:11.908 779-855/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [2, 0]
2021-06-28 19:09:11.988 753-753/? E/mm-camera: <STATS ><ERROR> 3068: stats_port_check_caps_reserve: Invalid Port capability type!
2021-06-28 19:09:11.988 753-753/? E/mm-camera: <STATS ><ERROR> 3068: stats_port_check_caps_reserve: Invalid Port capability type!
2021-06-28 19:09:12.025 17592-20075/com.emmaprod W/OpenGLRenderer: Surface doesn't have any previously queued frames, nothing to readback from
2021-06-28 19:09:12.043 17592-17592/com.emmaprod E/SurfaceViewImpl: PreviewView.SurfaceViewImplementation.getBitmap() failed with error 3
2021-06-28 19:09:12.059 753-753/? E/libc: Access denied finding property "ro.camera.req.fmq.size"
2021-06-28 19:09:12.062 753-753/? E/libc: Access denied finding property "ro.camera.res.fmq.size"
2021-06-28 19:09:12.100 753-3515/? E/mm-camera: <SENSOR><ERROR> 203: module_sensor_offload_init_config: tom han module_sensor_offload SENSOR_INIT success
2021-06-28 19:09:12.108 753-753/? E/CamComm1.0-MD: Mismatched tag type when updating entry enable (-2146762752) of type byte; got type int32 data instead 
2021-06-28 19:09:12.108 753-753/? E/CamComm1.0-MD: Mismatched tag type when updating entry is_main (-2146762751) of type byte; got type int32 data instead 
2021-06-28 19:09:12.114 953-24003/? W/CameraDeviceClient: createSurfaceFromGbp: Camera 0 with consumer usage flag: 2304: Forcing asynchronous mode for stream
2021-06-28 19:09:12.115 953-24003/? W/CameraDeviceClient: createSurfaceFromGbp: Camera 0: Overriding format 0x4 to IMPLEMENTATION_DEFINED
2021-06-28 19:09:12.117 753-3515/? E/mm-camera: <SENSOR><ERROR> 246: module_sensor_offload_init_config: tom han module_sensor_offload is_insensor success
2021-06-28 19:09:12.133 753-3515/? E/mm-camera: <SENSOR><ERROR> 37: af_actuator_power_up: failed: rc = -1
2021-06-28 19:09:12.133 753-3515/? E/mm-camera: <SENSOR><ERROR> 710: af_actuator_init: failed rc -5
2021-06-28 19:09:12.133 753-3515/? E/mm-camera: <SENSOR><ERROR> captain_hi1336_back_hlt_i_autofocus_calibration: 491: [af otp] marco: 593, infinity: 143
2021-06-28 19:09:12.133 753-3515/? E/mm-camera: <SENSOR><ERROR> captain_hi1336_back_hlt_i_autofocus_calibration: 507: [ af otp] adjusted code_per_step: 121, qvalue: 128
2021-06-28 19:09:12.136 753-3515/? E/mm-camera: <SENSOR><ERROR> 293: module_sensor_offload_init_config: tom han module_sensor_offload AEC success
2021-06-28 19:09:12.137 753-3515/? E/mm-camera: <SENSOR><ERROR> 312: module_sensor_offload_init_config: tom han module_sensor_offload all Success
2021-06-28 19:09:12.155 753-3503/? E/mm-camera: <IMGLIB><ERROR> 322: module_depth_map_handle_ctrl_parm: E
2021-06-28 19:09:12.155 753-3503/? E/mm-camera: <IMGLIB><ERROR> 430: module_depth_map_set_config_param: X
2021-06-28 19:09:12.155 753-3503/? E/mm-camera: <IMGLIB><ERROR> 254: module_depth_map_send_config_data: CAM_STREAM_TYPE_DEPTH not set !!
2021-06-28 19:09:12.155 753-3503/? E/mm-camera: <IMGLIB><ERROR> 369: module_depth_map_handle_ctrl_parm: Error module_depth_map_send_config_data
2021-06-28 19:09:12.155 753-3503/? E/mm-camera: <IMGLIB><ERROR> 373: module_depth_map_handle_ctrl_parm: X
2021-06-28 19:09:12.155 753-3503/? E/mm-camera: <IMGLIB><ERROR> 890: module_imgbase_process_param: [depth_map] handle_ctrl_parm error : type=10, rc=-5
2021-06-28 19:09:12.159 753-3503/? W/mm-camera: <IFACE >< WARN> 2857: iface_util_modify_plane_info_for_native_buf: iface_util_modify_plane_info_for_native_buf: Invalid cam_format -2088533117
2021-06-28 19:09:12.159 753-3503/? W/mm-camera: <IFACE >< WARN> 2892: iface_util_calculate_frame_length_for_native_buf: iface_util_calculate_frame_length_for_native_buf: returned fail=-1
2021-06-28 19:09:12.159 753-3503/? W/mm-camera: <IFACE >< WARN> 2857: iface_util_modify_plane_info_for_native_buf: iface_util_modify_plane_info_for_native_buf: Invalid cam_format -2088533117
2021-06-28 19:09:12.159 753-3503/? W/mm-camera: <IFACE >< WARN> 2892: iface_util_calculate_frame_length_for_native_buf: iface_util_calculate_frame_length_for_native_buf: returned fail=-1
2021-06-28 19:09:12.173 753-3503/? W/mm-camera: <IMGLIB>< WARN> 5728: faceproc_hw_comp_create: Library not loaded, handle=0x0, p_ops=0xd8c4d04c
2021-06-28 19:09:12.173 753-3503/? W/mm-camera: <IMGLIB>< WARN> 3898: module_faceproc_comp_create: create comp failed, eng_type=0
2021-06-28 19:09:12.173 753-3503/? W/mm-camera: <IMGLIB>< WARN> 1060: faceproc_dsp_comp_create: fddsp stub library not loaded
2021-06-28 19:09:12.173 753-3503/? W/mm-camera: <IMGLIB>< WARN> 3898: module_faceproc_comp_create: create comp failed, eng_type=1
2021-06-28 19:09:12.184 753-753/? W/CAM_MctServ_3: type=1400 audit(0.0:22000): avc: denied { read } for name="u:object_r:system_prop:s0" dev="tmpfs" ino=2242 scontext=u:r:hal_camera_default:s0 tcontext=u:object_r:system_prop:s0 tclass=file permissive=0
2021-06-28 19:09:12.198 753-3503/? E/libc: Access denied finding property "sys.camera.miui.apk"
2021-06-28 19:09:12.199 753-3503/? E/mm-camera: <SENSOR><ERROR> 8122: module_sensor_set_decode_format_based_on_pack_mode: Pack mode is 0
2021-06-28 19:09:12.199 753-1589/? E/quadracfa_dummy: inside dummy remosaic_init 1
2021-06-28 19:09:12.199 753-1589/? E/quadracfa_dummy: inside dummy remosaic_gainmap_gen 1
2021-06-28 19:09:12.219 753-3525/? E/mm-camera: <STATS_AF ><ERROR> 596: af_haf_default_mixer_active_algo_selection: ERROR! Unable to find active mixer algo !! 
2021-06-28 19:09:12.219 753-3503/? E/mm-camera: <IFACE ><ERROR> 2269: iface_util_set_chromatix: iface_util_set_chromatix:2269 failed: iface_hvx_open rc 0
2021-06-28 19:09:12.222 753-3524/? E/mm-camera: <STATS_AEC ><ERROR> 579: aec_led_cal_apply_calibration: Calibrate data size 0 not expected with tuning size: 0
2021-06-28 19:09:12.233 753-3541/? E/mm-still: omx_component_set_callbacks: Bad Parameter
2021-06-28 19:09:12.277 753-753/? E/QCamera: <HAL><ERROR> translateToHalMetadata: 15900: tong mHmdZOOMEable = 0
2021-06-28 19:09:12.290 753-2256/? E/QCamera: <HAL><ERROR> translateToHalMetadata: 15900: tong mHmdZOOMEable = 0
2021-06-28 19:09:12.296 753-2256/? E/QCamera: <HAL><ERROR> translateToHalMetadata: 15900: tong mHmdZOOMEable = 0
2021-06-28 19:09:12.809 1644-24165/? W/ActivityManager: Scheduling restart of crashed service com.android.chrome/org.chromium.chrome.browser.customtabs.CustomTabsConnectionService in 1000ms
2021-06-28 19:09:12.813 1644-24155/? W/ActivityManager: Scheduling restart of crashed service com.hmdglobal.app.midtest/.NvDealService in 10996ms
2021-06-28 19:09:12.821 1644-24155/? W/ActivityManager: Scheduling restart of crashed service com.qualcomm.qti.callenhancement/.CallEnhancementService in 20988ms
2021-06-28 19:09:12.832 1644-1695/? W/ActivityManager: Scheduling restart of crashed service com.qualcomm.qti.qdma/.job.QDMAJobService in 30977ms
2021-06-28 19:09:12.832 1644-1695/? W/ActivityManager: Scheduling restart of crashed service com.qualcomm.qti.qdma/.app.ActiveCareService in 40977ms
2021-06-28 19:09:12.832 1644-1695/? W/ActivityManager: Scheduling restart of crashed service com.qualcomm.qti.qdma/.uploader.UploaderService in 50977ms
2021-06-28 19:09:12.841 1644-1695/? W/ActivityManager: Scheduling restart of crashed service com.qualcomm.simcontacts/.SimContactsService in 60968ms
2021-06-28 19:09:12.843 1644-2773/? W/ActivityManager: Scheduling restart of crashed service com.qualcomm.qti.workloadclassifier/.WLCService in 70966ms
2021-06-28 19:09:13.634 17592-3578/com.emmaprod W/StreamingFormatChecker: ML Kit has detected that you seem to pass camera frames to the detector as a Bitmap object. This is inefficient. Please use YUV_420_888 format for camera2 API or NV21 format for (legacy) camera API and directly pass down the byte array to ML Kit.
2021-06-28 19:09:14.309 3579-3579/? E/.android.chrom: Not starting debugger since process cannot load the jdwp agent.
2021-06-28 19:09:14.555 3579-3579/? W/.android.chrom: JIT profile information will not be recorded: profile file does not exits.
2021-06-28 19:09:14.555 3579-3579/? W/.android.chrom: JIT profile information will not be recorded: profile file does not exits.
2021-06-28 19:09:14.837 3579-3613/? W/DynamiteModule: Local module descriptor class for com.google.android.gms.googlecertificates not found.
2021-06-28 19:09:14.941 1644-24153/? W/ActivityManager: Scheduling restart of crashed service com.android.chrome/org.chromium.chrome.browser.customtabs.CustomTabsConnectionService in 78868ms
2021-06-28 19:09:18.635 17592-3604/com.emmaprod W/StreamingFormatChecker: ML Kit has detected that you seem to pass camera frames to the detector as a Bitmap object. This is inefficient. Please use YUV_420_888 format for camera2 API or NV21 format for (legacy) camera API and directly pass down the byte array to ML Kit.
2021-06-28 19:09:20.299 1644-1691/? W/System: A resource failed to call close. 
2021-06-28 19:09:20.299 1644-1691/? W/System: A resource failed to call close. 
2021-06-28 19:09:20.300 1644-1691/? W/System: A resource failed to call release. 
2021-06-28 19:09:20.300 1644-1691/? W/System: A resource failed to call close. 
2021-06-28 19:09:20.300 1644-1691/? W/System: A resource failed to call close. 
2021-06-28 19:09:23.663 17592-3490/com.emmaprod W/StreamingFormatChecker: ML Kit has detected that you seem to pass camera frames to the detector as a Bitmap object. This is inefficient. Please use YUV_420_888 format for camera2 API or NV21 format for (legacy) camera API and directly pass down the byte array to ML Kit.
2021-06-28 19:09:23.875 3623-3623/? E/bal.app.midtes: Not starting debugger since process cannot load the jdwp agent.
2021-06-28 19:09:24.265 3623-3623/? W/ContextImpl: Calling a method in the system process without a qualified user: android.app.ContextImpl.startService:1570 android.content.ContextWrapper.startService:669 com.longcheertel.fmradio.FMAdapterApp.onCreate:81 android.app.Instrumentation.callApplicationOnCreate:1190 android.app.ActivityThread.handleBindApplication:6479
G0retZ commented 3 years ago

And Android camera component looks like this:

import React, { useMemo, useRef } from 'react';
import { StyleSheet, NativeModules } from 'react-native';
import { Camera, useCameraDevices } from 'react-native-vision-camera';

import useAppState from '../../../hooks/useAppState';
import { useCameraPermissions } from '../hooks';
import { PermissionStatus } from '../types';
import useInterval from '../../../hooks/useInterval';

const qrcodeLocalImage = NativeModules.QRCodeLocalImage;

type Props = {
  visible: boolean;
  flash: boolean;
  onScannedCodes?: (codes: string[]) => void;
};

export default ({ visible, flash, onScannedCodes }: Props) => {
  const camera = useRef<Camera>(null);
  const { status } = useCameraPermissions();
  const devices = useCameraDevices();
  const appState = useAppState();
  const device = useMemo(
    () => (status === PermissionStatus.allowed ? devices.back : undefined),
    [devices, status]
  );
  const isForeground = useMemo(() => visible && appState === 'active', [
    visible,
    appState,
  ]);

  useInterval(
    async () => {
      if (camera.current && visible) {
        try {
          const photo = await camera.current?.takeSnapshot({
            quality: 80,
            skipMetadata: true,
          });

          if (photo && qrcodeLocalImage) {
            onScannedCodes?.(await qrcodeLocalImage.decode(photo.path));
          }
        } catch (e) {
          // NoOp
        }
      }
    },
    isForeground && device ? 500 : undefined
  );

  return device ? (
    <Camera
      ref={camera}
      style={StyleSheet.absoluteFill}
      device={device}
      torch={isForeground && flash ? 'on' : 'off'}
      isActive={isForeground}
      photo
    />
  ) : null;
};
mrousavy commented 3 years ago

Oh okay, so the issue is that the frame processor is not being set on the first render? The title of the issue is a bit misleading. "Camera start and stop issues" sounds like the camera blackscreens.

Anyways, I think I already found a fix. I haven't tested the change, but here's the diff: https://github.com/mrousavy/react-native-vision-camera/commit/b10b2c10fc6084f13a3a667b970910c0d7541b34

I have released this (and Frame Processors for Android! 🎉) under the beta tag on npm (see Release 2.4.2-beta.3).

$ npm i react-native-vision-camera@beta
G0retZ commented 3 years ago

But what about camera not stopping on view unmount? I can record a video of this...

G0retZ commented 3 years ago

Will be happy to get Android Frame Processor also 🎉

mrousavy commented 3 years ago

I can record a video of this... Please do, because I cannot reproduce it in the example app.

Add the following code to your page:

useEffect(() => {
  console.log("mounted")
  return () => {
    console.log("unmounted!")
  }
}, [])

, then start the app clean (no fast-refresh) and showcase the issue. Make sure unmounted gets logged.

G0retZ commented 3 years ago

@mrousavy so here it the record https://youtube.com/shorts/Yzv0afk7aJ4?feature=share

The green indicator dot on the top right of the screen stays after going back. And never goes off. If I send app to background it disappears. But appears back once app gets resumed.

Here are the logs:

 LOG  mounted
 WARN  [native] VisionCamera.captureOutput(_:didDrop:from:): Dropped a Frame. This might indicate that your Frame Processor is doing too much work. Either throttle the frame processor's frame rate, or optimize your frame processor's execution speed. Frame drop reason: FrameWasLate
 LOG  unmounted!

Visually unmounted log happened after the view was animated to go offscreen.

G0retZ commented 3 years ago

Also will try to update the lib, then will report on it's behavior

mrousavy commented 3 years ago

Hm, that's really weird. Maybe there's a memory leak? Are you using react-navigation or react-native-navigation?

G0retZ commented 3 years ago

We are using react-navigation

G0retZ commented 3 years ago

Tried with the latest 2.4.2-beta.7 The issue is still here. @mrousavy You mentioned something about the leak. How there can be a leak if the component is unmounted? 🤔

G0retZ commented 3 years ago

Also, looks like this fix actually doesn't help #b10b2c1 I need a delay anyway to let frame processor work. Actually it looks like frame processor gets stuck after 1 run 🤔 And start to work normally only after disabling CameraView and enabling again.

G0retZ commented 3 years ago

View with CameraView mounted and before CameraView disabled (Frame processor called only once):

2021-06-29 15:27:02.643754+0300 emma dev[2737:1147307] FrameProcessorBindings: Setting new frame processor...
2021-06-29 15:27:02.643803+0300 emma dev[2737:1147307] FrameProcessorBindings: Adapting Shareable value from function (conversion to worklet)...
2021-06-29 15:27:02.643914+0300 emma dev[2737:1147307] FrameProcessorBindings: Successfully created worklet!
2021-06-29 15:27:02.654980+0300 emma dev[2737:1147299] FrameProcessorBindings: Converting worklet to Objective-C callback...
2021-06-29 15:27:02.655318+0300 emma dev[2737:1147299] FrameProcessorBindings: Frame processor set!
2021-06-29 15:27:02.657990+0300 emma dev[2737:1147134] [native] VisionCamera.didSetProps(_:): Updating 13 prop(s)...
2021-06-29 15:27:02.658195+0300 emma dev[2737:1147551] [native] VisionCamera.configureCaptureSession(): Configuring Session...
2021-06-29 15:27:02.658223+0300 emma dev[2737:1147551] [native] VisionCamera.configureCaptureSession(): Initializing Camera with device com.apple.avfoundation.avcapturedevice.built-in_video:6...
2021-06-29 15:27:02.658256+0300 emma dev[2737:1147551] [native] VisionCamera.configureCaptureSession(): Adding Video input...
2021-06-29 15:27:02.662385+0300 emma dev[2737:1147551] [native] VisionCamera.configureCaptureSession(): Adding Photo output...
2021-06-29 15:27:02.663920+0300 emma dev[2737:1147551] [native] VisionCamera.configureCaptureSession(): Adding Video Data output...
2021-06-29 15:27:02.664479+0300 emma dev[2737:1147551] [native] VisionCamera.invokeOnInitialized(): Camera initialized!
2021-06-29 15:27:02.664535+0300 emma dev[2737:1147551] [native] VisionCamera.configureCaptureSession(): Session successfully configured!
2021-06-29 15:27:02.665129+0300 emma dev[2737:1147551] [native] VisionCamera.configureFormat(): Configuring Format...
2021-06-29 15:27:02.665137+0300 emma dev[2737:1147551] [native] VisionCamera.configureDevice(): Configuring Device...
2021-06-29 15:27:02.665144+0300 emma dev[2737:1147551] [native] VisionCamera.configureDevice(): Device successfully configured!
2021-06-29 15:27:02.665277+0300 emma dev[2737:1147551] [native] VisionCamera.didSetProps(_:): Starting Session...
2021-06-29 15:27:03.259933+0300 emma dev[2737:1147551] [native] VisionCamera.didSetProps(_:): Started Session!
2021-06-29 15:27:10.769634+0300 emma dev[2737:1147307] FrameProcessorBindings: Setting new frame processor...
2021-06-29 15:27:10.769747+0300 emma dev[2737:1147307] FrameProcessorBindings: Adapting Shareable value from function (conversion to worklet)...
2021-06-29 15:27:10.769829+0300 emma dev[2737:1147307] FrameProcessorBindings: Successfully created worklet!
2021-06-29 15:27:10.769945+0300 emma dev[2737:1147296] FrameProcessorBindings: Converting worklet to Objective-C callback...
2021-06-29 15:27:10.770171+0300 emma dev[2737:1147296] FrameProcessorBindings: Frame processor set!
2021-06-29 15:27:10.794642+0300 emma dev[2737:1147296] Frame processor called!
2021-06-29 15:27:10.795332+0300 emma dev[2737:1147296] Metal API Validation Enabled
2021-06-29 15:27:10.856776+0300 emma dev[2737:1147134] [native] VisionCamera.didSetProps(_:): Updating 1 prop(s)...
2021-06-29 15:27:10.857034+0300 emma dev[2737:1147554] [native] VisionCamera.didSetProps(_:): Stopping Session...
2021-06-29 15:27:10.985358+0300 emma dev[2737:1147554] [native] VisionCamera.didSetProps(_:): Stopped Session!
G0retZ commented 3 years ago

Logs after CameraView reenabled. Frame processor gets called constantly:

2021-06-29 15:27:15.703789+0300 emma dev[2737:1147134] [native] VisionCamera.didSetProps(_:): Updating 1 prop(s)...
2021-06-29 15:27:15.704219+0300 emma dev[2737:1147296] [native] VisionCamera.didSetProps(_:): Starting Session...
2021-06-29 15:27:16.259640+0300 emma dev[2737:1147296] [native] VisionCamera.didSetProps(_:): Started Session!
2021-06-29 15:27:16.261332+0300 emma dev[2737:1147296] Frame processor called!
2021-06-29 15:27:16.782095+0300 emma dev[2737:1147556] Frame processor called!
2021-06-29 15:27:17.294113+0300 emma dev[2737:1147556] Frame processor called!
2021-06-29 15:27:17.825266+0300 emma dev[2737:1147296] Frame processor called!
2021-06-29 15:27:18.330277+0300 emma dev[2737:1147296] Frame processor called!
2021-06-29 15:27:18.857436+0300 emma dev[2737:1147296] Frame processor called!
2021-06-29 15:27:19.361439+0300 emma dev[2737:1147296] Frame processor called!
2021-06-29 15:27:19.893778+0300 emma dev[2737:1147624] Frame processor called!
2021-06-29 15:27:20.426882+0300 emma dev[2737:1147300] Frame processor called!
2021-06-29 15:27:20.529367+0300 emma dev[2737:1147134] [native] VisionCamera.didSetProps(_:): Updating 1 prop(s)...
2021-06-29 15:27:20.529628+0300 emma dev[2737:1147300] [native] VisionCamera.didSetProps(_:): Stopping Session...
2021-06-29 15:27:20.695193+0300 emma dev[2737:1147300] [native] VisionCamera.didSetProps(_:): Stopped Session!
2021-06-29 15:27:22.628997+0300 emma dev[2737:1147307] FrameProcessorBindings: Removing frame processor...
2021-06-29 15:27:22.629324+0300 emma dev[2737:1147134] FrameProcessorBindings: Frame processor removed!
mrousavy commented 3 years ago

Can you reproduce this in the example app?

G0retZ commented 3 years ago

Trying to build this one https://github.com/mrousavy/vision-camera-image-labeler/tree/main/example It fails... But what I see so far, there is no case where CameraView gets unmounted while app is on the screen. Once build, will add this case and check if it reproduces...

G0retZ commented 3 years ago

Analyzing the difference in logs between first deactivate than unmount and unmount while was active. The case with the issue lacks these logs:

[2861:1181594] [native] VisionCamera.didSetProps(_:): Stopping Session...
[2861:1181594] [native] VisionCamera.didSetProps(_:): Stopped Session!

But both has these

[2861:1181425] FrameProcessorBindings: Removing frame processor...
[2861:1181241] FrameProcessorBindings: Frame processor removed!
G0retZ commented 3 years ago

@mrousavy thank you a lot with your help! I finally managed to make example app build and run (it's a bit outdated and not fully set), replicated the case with mount/unmount and saw no issue with camera hang I reported. Looks like we have a leak somewhere... I altered the example code a bit to test:

import * as React from 'react';
import { TouchableOpacity, Text } from 'react-native';

import { StyleSheet, View, ActivityIndicator } from 'react-native';
import {
  useCameraDevices,
  useFrameProcessor,
} from 'react-native-vision-camera';
import { Camera } from 'react-native-vision-camera';
import { labelImage } from 'vision-camera-image-labeler';

export default function App() {
  const [hasPermission, setHasPermission] = React.useState(false);
  const [active, setActive] = React.useState(true);
  const [mount, setMount] = React.useState(true);
  const devices = useCameraDevices('wide-angle-camera');
  const device = devices.back;

  React.useEffect(() => {
    (async () => {
      const status = await Camera.requestCameraPermission();
      setHasPermission(status === 'authorized');
    })();
  }, []);

  const frameProcessor = useFrameProcessor((frame) => {
    const labels = labelImage(frame);
    _log(`Labels: ${labels.join(', ')}`);
  }, []);

  return (
    <View style={styles.container}>
      {device != null && mount && hasPermission ? (
        <Camera
          style={StyleSheet.absoluteFill}
          device={device}
          isActive={active}
          frameProcessor={frameProcessor}
          frameProcessorFps={5}
        />
      ) : (
        <ActivityIndicator size="large" color="white" />
      )}
      <TouchableOpacity
        style={styles.button}
        onPress={() => {
          setActive(!active);
        }}
      >
        <Text style={{ fontSize: 24 }}>
          {active ? 'Deactivate' : 'Activate'}
        </Text>
      </TouchableOpacity>
      <TouchableOpacity
        style={styles.button}
        onPress={() => {
          setMount(!mount);
        }}
      >
        <Text style={{ fontSize: 24 }}>{mount ? 'Unmount' : 'Mount'}</Text>
      </TouchableOpacity>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    alignItems: 'center',
    justifyContent: 'center',
  },
  button: {
    padding: 24,
    margin: 24,
    backgroundColor: 'rgba(255,255,255,0.7)',
  },
  box: {
    width: 60,
    height: 60,
    marginVertical: 20,
  },
});
mrousavy commented 3 years ago

Hey @G0retZ! Can you check if https://github.com/mrousavy/react-native-vision-camera/pull/265 fixes the issue for you?

G0retZ commented 3 years ago

Hey @mrousavy! Sorry for a long time to answer, we abandoned camera feature in our app but now we returned to it. And yes, now it works great (2.5.0)! Thanks a lot! 🙏

vzhovnitsky commented 2 years ago

Having same issue with "react-native-vision-camera": "^2.12.0", Using it with react-navigation. Green dot stays even after I close the screen with Camera

const isFocused = useIsFocused();
const devices = useCameraDevices();
const device = devices.back;
const [frameProcessor, barcodes] = useScanBarcodes([BarcodeFormat.QR_CODE]);

{device && (
                <Camera
                    style={StyleSheet.absoluteFill}
                    device={device}
                    isActive={isFocused}
                    frameProcessor={frameProcessor}
                    frameProcessorFps={5}
                    torch={flashOn ? 'on' : 'off'}
                />
)}
arthwood commented 2 years ago

I have the same issue but Android only. The camera stays active as soon as I visit this screen and persists through whole app session until it's moved to background:

import { isDefined, Nullable } from '@core'
import { useIsFocused } from '@react-navigation/native'
import React, { FC, useEffect, useMemo, useRef, useState } from 'react'
import { Text, View } from 'react-native'
import { Camera, CameraDevice, CameraPermissionStatus, useCameraDevices } from 'react-native-vision-camera'

interface CameraState {
  cameraDevice?: CameraDevice
  cameraPermissionStatus?: CameraPermissionStatus
}

interface CameraReadyState extends CameraState {
  cameraDevice: CameraDevice
  cameraPermissionStatus: 'authorized'
}

const isCameraStateReady = (cameraState: CameraState): cameraState is CameraReadyState =>
  isDefined(cameraState.cameraDevice) && cameraState.cameraPermissionStatus === 'authorized'

const PictureScreen: FC = () => {
  const isFocused = useIsFocused()
  const cameraRef = useRef<Nullable<Camera>>(null)
  const { back: cameraDevice } = useCameraDevices()
  const [cameraPermissionStatus, setCameraPermissionStatus] = useState<CameraPermissionStatus>()

  const cameraState = useMemo<CameraState>(() => ({ cameraDevice, cameraPermissionStatus }), [
    cameraDevice,
    cameraPermissionStatus,
  ])
  const isCameraAuthorized = cameraPermissionStatus === 'authorized'

  useEffect(() => {
    Camera.getCameraPermissionStatus().then(setCameraPermissionStatus)
  }, [setCameraPermissionStatus])

  useEffect(() => {
    if (!isCameraAuthorized) Camera.requestCameraPermission().then(setCameraPermissionStatus)
  }, [isCameraAuthorized, setCameraPermissionStatus])

  return (
    <View>
      {isCameraStateReady(cameraState) ? (
        <Camera audio={false} device={cameraState.cameraDevice} isActive={isFocused} photo ref={cameraRef} video />
      ) : (
        <View>
          <Text>Loading camera...</Text>
        </View>
      )}
    </View>
  )
}

isFocused is properly changing from true to false when navigating from this screen and isCameraStateReady(cameraState) is still true. I'm using react-navigation v6, but the dot is still active when I navigate to next screen in the same stack.

mrousavy commented 11 months ago

Hey! I just found out that I really forgot to close and dispose the locked Camera resources, so I just fixed that in this PR: https://github.com/mrousavy/react-native-vision-camera/pull/2174 Now the Camera fully & synchronously closes all resources (CameraSession, CameraDevice, OpenGL context, Video & Photo outputs, Photo Synchronizer) once the view gets removed from the React view hierarchy ("unmounted"), and things like Flash, Torch, NFC, and other Camera components should be working again.

There is still a small issue that causes once Camera component to turn into a blackscreen when navigating back and forth between two Camera components, that's a pretty rare edge case but I will still try to fix that soon when I have some free time.

If you appreciate my time, expertise and dedication to this project, pleas 💖 consider sponsoring me on GitHub 💖 to support the development of this project.