🐛 Unable to use more than 3 video outputs on Android

santiagofm commented 2 weeks ago

What's happening?

When I try to use more than 3 camera outputs on Android it throws an exception:

Camera.onError(unknown/unknown): No supported surface combination is found for camera device - Id : 0.  May be attempting to bind too many use cases. Existing surfaces: [] New configs: [androidx.camera.core.impl.ImageAnalysisConfig@5500c94, androidx.camera.core.impl.ImageCaptureConfig@2ce66a6, androidx.camera.core.streamsharing.StreamSharingConfig@6ea4732, androidx.camera.core.impl.ImageAnalysisConfig@f8c78e7]

I'm using the example app code for the code scanner use case and adding a few more "video outputs" for running a frame processor & being able to take photos programatically.

I need to be able to run 4 "video ouputs" at the same time: preview the camera, take photos programatically, scan codes using the built-in code-scanner & run a frame processor:

Reproduceable Code

import * as React from 'react'
import { useCallback } from 'react'
import { StyleSheet, View } from 'react-native'
import type { Code } from 'react-native-vision-camera'
import { useCameraDevice, useCodeScanner, useFrameProcessor } from 'react-native-vision-camera'
import { Camera } from 'react-native-vision-camera'
import { useIsForeground } from './hooks/useIsForeground'
import { StatusBarBlurBackground } from './views/StatusBarBlurBackground'
import { useIsFocused } from '@react-navigation/core'

export function CodeScannerPage(): React.ReactElement {
  // 1. Use a simple default back camera
  const device = useCameraDevice('back')

  // 2. Only activate Camera when the app is focused and this screen is currently opened
  const isFocused = useIsFocused()
  const isForeground = useIsForeground()
  const isActive = isFocused && isForeground

  // 4. On code scanned, console.log
  const onCodeScanned = useCallback((codes: Code[]) => {
    console.log(`Scanned ${codes.length} codes:`, codes)
    const value = codes[0]?.value
    if (value == null) return

    console.log(value)
  }, [])

  // 5. Initialize the Code Scanner to scan QR codes and Barcodes
  const codeScanner = useCodeScanner({
    codeTypes: ['qr', 'ean-13'],
    onCodeScanned: onCodeScanned,
  })

  const frameProcessor = useFrameProcessor((frame) => {
    // nothing to do here
  }, [])

  return (
    <View style={styles.container}>
      {device != null && (
        <Camera
          style={StyleSheet.absoluteFill}
          device={device}
          isActive={isActive}
          photo={true}
          codeScanner={codeScanner}
          enableZoomGesture={true}
          frameProcessor={frameProcessor}
        />
      )}
      <StatusBarBlurBackground />
    </View>
  )
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: 'black',
  },
})

Relevant log output

2024-05-14 13:19:03.303  2943-3123  ReactNativeJS           com.mrousavy.camera.example          I  29879920671: 1280x720 yuv Frame (landscape-left)
2024-05-14 13:19:03.303  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D  1280 x 720 Image with format #35. Logging 5 parameters:
2024-05-14 13:19:03.303  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> 42.0 (java.lang.Double)
2024-05-14 13:19:03.303  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> true (java.lang.Boolean)
2024-05-14 13:19:03.303  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> {test=0.0, second=test} (java.util.HashMap)
2024-05-14 13:19:03.303  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> hello! (java.lang.String)
2024-05-14 13:19:03.303  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> [another test, 5.0] (java.util.ArrayList)
2024-05-14 13:19:03.383  2943-2943  VisionCameraProxy       com.mrousavy.camera.example          D  Finding view 85...
2024-05-14 13:19:03.383  2943-2943  VisionCameraProxy       com.mrousavy.camera.example          D  Found view 85!
2024-05-14 13:19:03.395  2943-5106  CameraView              com.mrousavy.camera.example          I  invokeOnAverageFpsChanged(29.743589743589745)
2024-05-14 13:19:03.396  2943-2943  CameraSession           com.mrousavy.camera.example          I  Camera Lifecycle changed to CREATED!
2024-05-14 13:19:03.401  2943-2943  CameraView              com.mrousavy.camera.example          I  Updating CameraSession...
2024-05-14 13:19:03.416  2943-2943  chatty                  com.mrousavy.camera.example          I  uid=10166(com.mrousavy.camera.example) identical 2 lines
2024-05-14 13:19:03.416  2943-2943  CameraView              com.mrousavy.camera.example          I  Updating CameraSession...
2024-05-14 13:19:03.417  2943-2943  CameraSession           com.mrousavy.camera.example          I  configure { ... }: Waiting for lock...
2024-05-14 13:19:03.417  2943-2943  CameraView              com.mrousavy.camera.example          I  A new configure { ... } call arrived, aborting this one...
2024-05-14 13:19:03.417  2943-2943  CameraSession           com.mrousavy.camera.example          I  configure { ... }: Waiting for lock...
2024-05-14 13:19:03.417  2943-2943  CameraSession           com.mrousavy.camera.example          I  configure { ... }: Updating CameraSession Configuration... Difference(deviceChanged=false, outputsChanged=false, sidePropsChanged=false, isActiveChanged=true, locationChanged=true)
2024-05-14 13:19:03.417  2943-2943  CameraSession           com.mrousavy.camera.example          I  Camera Lifecycle changed to STARTED!
2024-05-14 13:19:03.417  2943-2943  ScreenFlashWrapper      com.mrousavy.camera.example          W  completePendingScreenFlashClear: none pending!
2024-05-14 13:19:03.417  2943-2943  DeferrableSurface       com.mrousavy.camera.example          D  surface closed,  useCount=1 closed=true androidx.camera.core.SurfaceRequest$2@c31e1f4
2024-05-14 13:19:03.417  2943-5073  Recorder                com.mrousavy.camera.example          D  Video source has transitioned to state: INACTIVE
2024-05-14 13:19:03.417  2943-2943  DeferrableSurface       com.mrousavy.camera.example          D  surface closed,  useCount=0 closed=true androidx.camera.core.processing.SurfaceEdge$SettableSurface@74a7948
2024-05-14 13:19:03.417  2943-2943  DeferrableSurface       com.mrousavy.camera.example          D  Surface terminated[total_surfaces=9, used_surfaces=7](androidx.camera.core.processing.SurfaceEdge$SettableSurface@74a7948}
2024-05-14 13:19:03.417  2943-2943  DeferrableSurface       com.mrousavy.camera.example          D  use count-1,  useCount=0 closed=true androidx.camera.core.SurfaceRequest$2@c31e1f4
2024-05-14 13:19:03.417  2943-2943  DeferrableSurface       com.mrousavy.camera.example          D  Surface no longer in use[total_surfaces=9, used_surfaces=6](androidx.camera.core.SurfaceRequest$2@c31e1f4}
2024-05-14 13:19:03.417  2943-2943  DeferrableSurface       com.mrousavy.camera.example          D  Surface terminated[total_surfaces=8, used_surfaces=6](androidx.camera.core.SurfaceRequest$2@c31e1f4}
2024-05-14 13:19:03.417  2943-2943  CameraSession           com.mrousavy.camera.example          I  Camera Lifecycle changed to CREATED!
2024-05-14 13:19:03.418  2943-2943  MetadataProvider        com.mrousavy.camera.example          I  Stopping location updates...
2024-05-14 13:19:03.418  2943-5113  VideoEncoderSession     com.mrousavy.camera.example          D  Surface can be closed: 55207761
2024-05-14 13:19:03.418  2943-5113  VideoEncoderSession     com.mrousavy.camera.example          D  closeInternal in READY state
2024-05-14 13:19:03.418  2943-5113  Recorder                com.mrousavy.camera.example          D  VideoEncoder can be released: androidx.camera.video.internal.encoder.EncoderImpl@2c930b7
2024-05-14 13:19:03.418  2943-5113  Recorder                com.mrousavy.camera.example          D  Transitioning streamId: 55207761 --> 0
2024-05-14 13:19:03.418  2943-2943  CameraSession           com.mrousavy.camera.example          I  configure { ... }: Completed CameraSession Configuration! (State: CREATED)
2024-05-14 13:19:03.418  2943-3222  Camera2CameraImpl       com.mrousavy.camera.example          D  {Camera@5ed3c9a[id=0]} Use cases [androidx.camera.core.ImageCapture-a1cfbad6-f368-44b5-a47d-6113a462dfda227621584, androidx.camera.core.ImageAnalysis-7439a59e-b42b-4a1e-b7c7-0a87e8717548158225870, androidx.camera.core.streamsharing.StreamSharing-5183cda8-6912-48ce-9301-7948c51bcbf0237509351] now DETACHED for camera
2024-05-14 13:19:03.418  2943-3222  UseCaseAttachState      com.mrousavy.camera.example          D  All use case: [] for camera: 0
2024-05-14 13:19:03.418  2943-2943  CameraSession           com.mrousavy.camera.example          I  configure { ... }: Waiting for lock...
2024-05-14 13:19:03.418  2943-3222  Camera2CameraImpl       com.mrousavy.camera.example          D  {Camera@5ed3c9a[id=0]} Resetting Capture Session
2024-05-14 13:19:03.419  2943-3222  SyncCaptureSessionImpl  com.mrousavy.camera.example          D  [androidx.camera.camera2.internal.SynchronizedCaptureSessionImpl@1bbafec] Session call close()
2024-05-14 13:19:03.419  2943-3123  ReactNativeJS           com.mrousavy.camera.example          I  29979936671: 1280x720 yuv Frame (landscape-left)
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  configure { ... }: Updating CameraSession Configuration... Difference(deviceChanged=true, outputsChanged=true, sidePropsChanged=true, isActiveChanged=true, locationChanged=true)
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  Creating new Outputs for Camera #0...
2024-05-14 13:19:03.419  2943-3222  Camera2CameraImpl       com.mrousavy.camera.example          D  {Camera@5ed3c9a[id=0]} Releasing session in state OPENED
2024-05-14 13:19:03.419  2943-3122  ExamplePlugin           com.mrousavy.camera.example          D  1280 x 720 Image with format #35. Logging 5 parameters:
2024-05-14 13:19:03.419  2943-3122  ExamplePlugin           com.mrousavy.camera.example          D    -> 42.0 (java.lang.Double)
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  Using FPS Range: null
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  Creating Preview output...
2024-05-14 13:19:03.419  2943-3122  ExamplePlugin           com.mrousavy.camera.example          D    -> true (java.lang.Boolean)
2024-05-14 13:19:03.419  2943-3122  ExamplePlugin           com.mrousavy.camera.example          D    -> {test=0.0, second=test} (java.util.HashMap)
2024-05-14 13:19:03.419  2943-3122  ExamplePlugin           com.mrousavy.camera.example          D    -> hello! (java.lang.String)
2024-05-14 13:19:03.419  2943-3122  ExamplePlugin           com.mrousavy.camera.example          D    -> [another test, 5.0] (java.util.ArrayList)
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  Creating Photo output...
2024-05-14 13:19:03.419  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D  1280 x 720 Image with format #35. Logging 5 parameters:
2024-05-14 13:19:03.419  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> 42.0 (java.lang.Double)
2024-05-14 13:19:03.419  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> true (java.lang.Boolean)
2024-05-14 13:19:03.419  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> {test=0.0, second=test} (java.util.HashMap)
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  Creating Video output...
2024-05-14 13:19:03.419  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> hello! (java.lang.String)
2024-05-14 13:19:03.419  2943-3122  ExampleKotlinPlugin     com.mrousavy.camera.example          D    -> [another test, 5.0] (java.util.ArrayList)
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  Creating new Recorder...
2024-05-14 13:19:03.419  2943-5113  Recorder                com.mrousavy.camera.example          D  Transitioning audio state: INITIALIZING --> INITIALIZING
2024-05-14 13:19:03.419  2943-5113  Recorder                com.mrousavy.camera.example          D  Releasing video encoder.
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  Creating YUV Frame Processor output...
2024-05-14 13:19:03.419  2943-2943  CameraSession           com.mrousavy.camera.example          I  Creating CodeScanner output...
2024-05-14 13:19:03.419  2943-5113  Recorder                com.mrousavy.camera.example          D  Releasing video encoder: androidx.camera.video.internal.encoder.EncoderImpl@2c930b7
2024-05-14 13:19:03.419  2943-5113  VideoEncoderSession     com.mrousavy.camera.example          D  VideoEncoder is releasing: androidx.camera.video.internal.encoder.EncoderImpl@2c930b7
2024-05-14 13:19:03.419  2943-5113  Recorder                com.mrousavy.camera.example          D  Transitioning Recorder internal state: IDLING --> CONFIGURING
2024-05-14 13:19:03.419  2943-3222  UseCaseAttachState      com.mrousavy.camera.example          D  Active and attached use case: [] for camera: 0
2024-05-14 13:19:03.420  2943-3222  UseCaseAttachState      com.mrousavy.camera.example          D  Active and attached use case: [] for camera: 0
2024-05-14 13:19:03.420  2943-5243  hw-BpHwBinder           com.mrousavy.camera.example          I  onLastStrongRef automatically unlinking death recipients
2024-05-14 13:19:03.420  2943-5113  VideoEncoder            com.mrousavy.camera.example          D  Transitioning encoder internal state: CONFIGURED --> RELEASED
2024-05-14 13:19:03.420  2943-5242  BpBinder                com.mrousavy.camera.example          I  onLastStrongRef automatically unlinking death recipients: android.media.IResourceManagerService
2024-05-14 13:19:03.420  2943-3222  Camera2CameraImpl       com.mrousavy.camera.example          D  {Camera@5ed3c9a[id=0]} Closing camera.
2024-05-14 13:19:03.420  2943-3222  Camera2CameraImpl       com.mrousavy.camera.example          D  {Camera@5ed3c9a[id=0]} Transitioning camera internal state: OPENED --> CLOSING
2024-05-14 13:19:03.421  2943-3222  CameraStateRegistry     com.mrousavy.camera.example          D  Recalculating open cameras:
                                                                                                    Camera                                       State                 
                                                                                                    -------------------------------------------------------------------
                                                                                                    Camera@5ed3c9a[id=0]                         CLOSING               
                                                                                                    Camera@d77ffd[id=1]                          UNKNOWN               
                                                                                                    -------------------------------------------------------------------
                                                                                                    Open count: 1 (Max allowed: 1)
2024-05-14 13:19:03.421  2943-3222  CameraStateMachine      com.mrousavy.camera.example          D  New public camera state CameraState{type=CLOSING, error=null} from CLOSING and null
2024-05-14 13:19:03.421  2943-3222  CameraStateMachine      com.mrousavy.camera.example          D  Publishing new public camera state CameraState{type=CLOSING, error=null}
2024-05-14 13:19:03.421  2943-3222  Camera2CameraImpl       com.mrousavy.camera.example          D  {Camera@5ed3c9a[id=0]} Resetting Capture Session
2024-05-14 13:19:03.421  2943-3222  Camera2CameraImpl       com.mrousavy.camera.example          D  {Camera@5ed3c9a[id=0]} Releasing session in state CLOSING
2024-05-14 13:19:03.421  2943-3222  SyncCaptureSessionImpl  com.mrousavy.camera.example          D  [androidx.camera.camera2.internal.SynchronizedCaptureSessionImpl@1bbafec] Session call super.close()
2024-05-14 13:19:03.421  2943-3222  UseCaseAttachState      com.mrousavy.camera.example          D  Active and attached use case: [] for camera: 0
2024-05-14 13:19:03.421  2943-3222  CaptureSession          com.mrousavy.camera.example          D  onSessionFinished()
2024-05-14 13:19:03.421  2943-3222  Camera2CameraImpl       com.mrousavy.camera.example          D  {Camera@5ed3c9a[id=0]} closing camera
2024-05-14 13:19:03.445  2943-2943  CameraSession           com.mrousavy.camera.example          I  Successfully created new Outputs for Camera #0!
2024-05-14 13:19:03.446  2943-2943  CameraSession           com.mrousavy.camera.example          I  Binding Camera #0...
2024-05-14 13:19:03.446  2943-5419  DynamiteModule          com.mrousavy.camera.example          I  Considering local module com.google.mlkit.dynamite.barcode:10000 and remote module com.google.mlkit.dynamite.barcode:0
2024-05-14 13:19:03.446  2943-5419  DynamiteModule          com.mrousavy.camera.example          I  Selected local version of com.google.mlkit.dynamite.barcode
2024-05-14 13:19:03.446  2943-2943  CameraSession           com.mrousavy.camera.example          I  Binding 5 use-cases...
2024-05-14 13:19:03.447  2943-2943  CameraOrientationUtil   com.mrousavy.camera.example          D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=90, isOppositeFacing=true, result=90
2024-05-14 13:19:03.447  2943-2943  CameraOrientationUtil   com.mrousavy.camera.example          D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=90, isOppositeFacing=true, result=90
2024-05-14 13:19:03.448  2943-2943  CapabilitiesByQuality   com.mrousavy.camera.example          D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=1, mediaType=audio/3gpp, bitrate=12200, sampleRate=8000, channels=1, profile=-1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=12000000, frameRate=30, width=1280, height=720, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-05-14 13:19:03.448  2943-2943  QualitySelector         com.mrousavy.camera.example          D  supportedQualities = [ConstantQuality{value=5, name=HD, typicalSizes=[1280x720]}]
2024-05-14 13:19:03.448  2943-2943  QualitySelector         com.mrousavy.camera.example          W  quality is not supported and will be ignored: ConstantQuality{value=6, name=FHD, typicalSizes=[1920x1080]}
2024-05-14 13:19:03.448  2943-2943  QualitySelector         com.mrousavy.camera.example          W  quality is not supported and will be ignored: ConstantQuality{value=4, name=SD, typicalSizes=[720x480, 640x480]}
2024-05-14 13:19:03.448  2943-2943  VideoCapture            com.mrousavy.camera.example          D  Found selectedQualities [ConstantQuality{value=5, name=HD, typicalSizes=[1280x720]}] by QualitySelector{preferredQualities=[ConstantQuality{value=6, name=FHD, typicalSizes=[1920x1080]}, ConstantQuality{value=5, name=HD, typicalSizes=[1280x720]}, ConstantQuality{value=4, name=SD, typicalSizes=[720x480, 640x480]}], fallbackStrategy=RuleStrategy{fallbackQuality=ConstantQuality{value=6, name=FHD, typicalSizes=[1920x1080]}, fallbackRule=1}}
2024-05-14 13:19:03.448  2943-2943  VideoCapture            com.mrousavy.camera.example          D  Set custom ordered resolutions = [1280x720]
2024-05-14 13:19:03.448  2943-2943  CameraOrientationUtil   com.mrousavy.camera.example          D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=90, isOppositeFacing=true, result=90
2024-05-14 13:19:03.449  2943-2943  DynamicRangeResolver    com.mrousavy.camera.example          D  Resolved dynamic range for use case androidx.camera.core.Preview-8e101cfa-7ae3-4016-8425-769a645a5365 to no compatible HDR dynamic ranges.
                                                                                                    DynamicRange@4a193e8{encoding=UNSPECIFIED, bitDepth=0}
                                                                                                    ->
                                                                                                    DynamicRange@5a4f90b{encoding=SDR, bitDepth=8}
2024-05-14 13:19:03.449  2943-2943  CapabilitiesByQuality   com.mrousavy.camera.example          D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=1, mediaType=audio/3gpp, bitrate=12200, sampleRate=8000, channels=1, profile=-1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=12000000, frameRate=30, width=1280, height=720, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-05-14 13:19:03.449  2943-2943  QualitySelector         com.mrousavy.camera.example          D  supportedQualities = [ConstantQuality{value=5, name=HD, typicalSizes=[1280x720]}]
2024-05-14 13:19:03.449  2943-2943  QualitySelector         com.mrousavy.camera.example          W  quality is not supported and will be ignored: ConstantQuality{value=6, name=FHD, typicalSizes=[1920x1080]}
2024-05-14 13:19:03.449  2943-2943  QualitySelector         com.mrousavy.camera.example          W  quality is not supported and will be ignored: ConstantQuality{value=4, name=SD, typicalSizes=[720x480, 640x480]}
2024-05-14 13:19:03.449  2943-2943  VideoCapture            com.mrousavy.camera.example          D  Found selectedQualities [ConstantQuality{value=5, name=HD, typicalSizes=[1280x720]}] by QualitySelector{preferredQualities=[ConstantQuality{value=6, name=FHD, typicalSizes=[1920x1080]}, ConstantQuality{value=5, name=HD, typicalSizes=[1280x720]}, ConstantQuality{value=4, name=SD, typicalSizes=[720x480, 640x480]}], fallbackStrategy=RuleStrategy{fallbackQuality=ConstantQuality{value=6, name=FHD, typicalSizes=[1920x1080]}, fallbackRule=1}}
2024-05-14 13:19:03.449  2943-2943  VideoCapture            com.mrousavy.camera.example          D  Set custom ordered resolutions = [1280x720]
2024-05-14 13:19:03.449  2943-2943  CameraOrientationUtil   com.mrousavy.camera.example          D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=90, isOppositeFacing=true, result=90
2024-05-14 13:19:03.449  2943-2943  CameraOrientationUtil   com.mrousavy.camera.example          D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=90, isOppositeFacing=true, result=90
2024-05-14 13:19:03.449  2943-2943  ResolutionsMerger       com.mrousavy.camera.example          D  The closer aspect ratio to the sensor size (1280x960) is 4/3.
2024-05-14 13:19:03.450  2943-2943  CameraOrientationUtil   com.mrousavy.camera.example          D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=90, isOppositeFacing=true, result=90
2024-05-14 13:19:03.450  2943-2943  CameraOrientationUtil   com.mrousavy.camera.example          D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=90, isOppositeFacing=true, result=90
2024-05-14 13:19:03.451  2943-5417  GooglePlayServicesUtil  com.mrousavy.camera.example          W  Google Play services out of date for com.mrousavy.camera.example.  Requires 203400000 but found 201817019
2024-05-14 13:19:03.451  2943-2943  CameraOrientationUtil   com.mrousavy.camera.example          D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=90, isOppositeFacing=true, result=90
2024-05-14 13:19:03.451  2943-5417  GoogleApiManager        com.mrousavy.camera.example          W  The service for com.google.android.gms.common.internal.service.zap is not available: ConnectionResult{statusCode=SERVICE_VERSION_UPDATE_REQUIRED, resolution=null, message=null}
2024-05-14 13:19:03.452  2943-2943  ResolutionsMerger       com.mrousavy.camera.example          D  Parent resolutions: [1280x960, 1280x720]
2024-05-14 13:19:03.453  2943-2943  CameraSession           com.mrousavy.camera.example          E  Failed to configure CameraSession! Error: No supported surface combination is found for camera device - Id : 0.  May be attempting to bind too many use cases. Existing surfaces: [] New configs: [androidx.camera.core.impl.ImageAnalysisConfig@5500c94, androidx.camera.core.impl.ImageCaptureConfig@2ce66a6, androidx.camera.core.streamsharing.StreamSharingConfig@6ea4732, androidx.camera.core.impl.ImageAnalysisConfig@f8c78e7], Config-Diff: Difference(deviceChanged=true, outputsChanged=true, sidePropsChanged=true, isActiveChanged=true, locationChanged=true)
                                                                                                    java.lang.IllegalArgumentException: No supported surface combination is found for camera device - Id : 0.  May be attempting to bind too many use cases. Existing surfaces: [] New configs: [androidx.camera.core.impl.ImageAnalysisConfig@5500c94, androidx.camera.core.impl.ImageCaptureConfig@2ce66a6, androidx.camera.core.streamsharing.StreamSharingConfig@6ea4732, androidx.camera.core.impl.ImageAnalysisConfig@f8c78e7]
                                                                                                        at androidx.camera.lifecycle.LifecycleCameraRepository.bindToLifecycleCamera(LifecycleCameraRepository.java:312)
                                                                                                        at androidx.camera.lifecycle.ProcessCameraProvider.bindToLifecycle(ProcessCameraProvider.java:609)
                                                                                                        at androidx.camera.lifecycle.ProcessCameraProvider.bindToLifecycle(ProcessCameraProvider.java:386)
                                                                                                        at com.mrousavy.camera.core.CameraSession.configureCamera(CameraSession.kt:433)
                                                                                                        at com.mrousavy.camera.core.CameraSession.configure(CameraSession.kt:168)
                                                                                                        at com.mrousavy.camera.react.CameraView$update$1.invokeSuspend(CameraView.kt:148)
                                                                                                        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                        at android.os.Handler.handleCallback(Handler.java:938)
                                                                                                        at android.os.Handler.dispatchMessage(Handler.java:99)
                                                                                                        at android.os.Looper.loop(Looper.java:223)
                                                                                                        at android.app.ActivityThread.main(ActivityThread.java:7656)
                                                                                                        at java.lang.reflect.Method.invoke(Native Method)
                                                                                                        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
                                                                                                        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
2024-05-14 13:19:03.453  2943-2943  CameraView              com.mrousavy.camera.example          E  invokeOnError(...):
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W  java.lang.IllegalArgumentException: No supported surface combination is found for camera device - Id : 0.  May be attempting to bind too many use cases. Existing surfaces: [] New configs: [androidx.camera.core.impl.ImageAnalysisConfig@5500c94, androidx.camera.core.impl.ImageCaptureConfig@2ce66a6, androidx.camera.core.streamsharing.StreamSharingConfig@6ea4732, androidx.camera.core.impl.ImageAnalysisConfig@f8c78e7]
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at androidx.camera.lifecycle.LifecycleCameraRepository.bindToLifecycleCamera(LifecycleCameraRepository.java:312)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at androidx.camera.lifecycle.ProcessCameraProvider.bindToLifecycle(ProcessCameraProvider.java:609)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at androidx.camera.lifecycle.ProcessCameraProvider.bindToLifecycle(ProcessCameraProvider.java:386)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at com.mrousavy.camera.core.CameraSession.configureCamera(CameraSession.kt:433)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at com.mrousavy.camera.core.CameraSession.configure(CameraSession.kt:168)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at com.mrousavy.camera.react.CameraView$update$1.invokeSuspend(CameraView.kt:148)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at android.os.Handler.handleCallback(Handler.java:938)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at android.os.Handler.dispatchMessage(Handler.java:99)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at android.os.Looper.loop(Looper.java:223)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at android.app.ActivityThread.main(ActivityThread.java:7656)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at java.lang.reflect.Method.invoke(Native Method)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
2024-05-14 13:19:03.453  2943-2943  System.err              com.mrousavy.camera.example          W      at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
2024-05-14 13:19:03.454  2943-2943  CameraSession           com.mrousavy.camera.example          I  configure { ... }: Waiting for lock...
2024-05-14 13:19:03.454  2943-2943  CameraSession           com.mrousavy.camera.example          I  Nothing changed, aborting configure { ... }

Camera Device

{
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "limited",
  "maxZoom": 10,
  "minZoom": 1,
  "maxExposure": 9,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "telephoto-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 0,
  "minExposure": -9,
  "name": "0 (BACK) androidx.camera.camera2",
  "hasFlash": false,
  "hasTorch": false,
  "position": "back",
  "id": "0"
}

Device

Pixel 6 API 30 emulator, also repo on real Xiaomi Redmi 13C

VisionCamera Version

main branch

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

[ ] I am using Expo
[X] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

santiagofm commented 2 weeks ago

While debugging I came accross the bindToLifecycle doc that clearly states:

...
Currently up to 3 use cases may be bound to a Lifecycle at any time. Exceeding capability of target camera device will throw an IllegalArgumentException.
...

So this issue seems to be an api limitation?

I can consistently repro this when I ahve more than 3 "video outputs" enabled at the same time(preview, photo, codescanner, frame processors) and it works fine when I remove any of them.

mrousavy commented 2 weeks ago

Can you try setting qualityPrioritization="speed"?

I built VisionCamera to not have that limitation (that was a huge issue in native Camera APIs, so it's hard to work around this issue)

santiagofm commented 2 weeks ago

Can you try setting qualityPrioritization="speed"?

I built VisionCamera to not have that limitation (that was a huge issue in native Camera APIs, so it's hard to work around this issue)

I guess you meant photoQualityBalance but yeah, same result.

Isn't it a limitation on the camera api itself?

...
Currently up to 3 use cases may be bound to a Lifecycle at any time. Exceeding capability of target camera device will throw an IllegalArgumentException.
...

santiagofm commented 2 weeks ago

@mrousavy Sorry if I'm missing something implementation wise, but it seems like we are assuming all devices will support binding any number of use cases— or at least that the camera api will disregard if not supported. Which doesn't seem to be the case and the camera API will just throw an exception.

Seems like we should be using the hardware_level of the device to determine the number of use cases that are supported? Not sure how the one thing translates to the other though.

I will try to fold the code scanner into my custom frame processor to reduce the number of use cases and workaround this. We cannot use multiple Camera components at the same time right? Even if one of them has the preview disabled?

mrousavy commented 2 weeks ago

but it seems like we are assuming all devices will support binding any number of use cases

No;

On iOS, I am explicitly checking if this is supported and Frame Processors and Video Capture have been carefully put together into a single video stream
On Android, CameraX already uses a similar feature under the hood called "stream sharing". So you could use any number of outputs here, although that feature is still in beta.

santiagofm commented 2 weeks ago

but it seems like we are assuming all devices will support binding any number of use cases

No;

On iOS, I am explicitly checking if this is supported and Frame Processors and Video Capture have been carefully put together into a single video stream

On Android, CameraX already uses a similar feature under the hood called "stream sharing". So you could use any number of outputs here, although that feature is still in beta.

Aha! Sorry, I didn't know that was a thing on Android(iOS works perfectly btw); unfortunately it doesn't seem to work as we would want it to :/

mrousavy commented 1 week ago

Hey - just created a PR to maybe fix this - can you test if that works for you? https://github.com/mrousavy/react-native-vision-camera/pull/2897 🙏

santiagofm commented 1 week ago

Hey - just created a PR to maybe fix this - can you test if that works for you? #2897 🙏

Will test it later today 🙌🏻 !

I ended up implementing the barcode recognizer with mlkit on a frame processor to reduce the number of outputs btw, that seems to work fine.

santiagofm commented 1 week ago

Hey - just created a PR to maybe fix this - can you test if that works for you? #2897 🙏

Hey! I just tested on a Xiaomi Redmi 13C and it's still throwing the error. I'm using 4 video outputs: preview, take photo programatically, built-in code scanner, frame processors.

mrousavy commented 1 week ago

Well then that means that the StreamSharing feature is not working properly on that specific phone.

We'd need to report that bug to Google/CameraX. I don't have the time to do that right now, so I'd really appreciate it if you took the time to do that (I mean it's also in your interest to get that fixed)

mrousavy / react-native-vision-camera