[camera] Missing logical cameras in list returned by `availableCameras()` on iOS.

orlys-ag commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues
[x] I have read the guide to filing a bug

Use case

Problem

The current platform-specific iOS implemention gets "all the available cameras" through:

NSMutableArray *discoveryDevices =
        [@[ AVCaptureDeviceTypeBuiltInWideAngleCamera, AVCaptureDeviceTypeBuiltInTelephotoCamera ]
            mutableCopy];
    if (@available(iOS 13.0, *)) {
      [discoveryDevices addObject:AVCaptureDeviceTypeBuiltInUltraWideCamera];
    }

This is missing logical cameras such as builtInDualCamera, builtInDualWideCamera, builtInTripleCamera. Currently, switching CameraDescription through setDescription has a noticeable delay.

Use case

I need access to the logical cameras (builtInDualCamera, builtInTripleCamera, etc.) so that I can take advantage of the platform's automatic hardware camera switch (i.e. going from ultra wide to wide lens when the zoom reaches 1.0).

DeviceType docs: https://developer.apple.com/documentation/avfoundation/avcapturedevice/devicetype

Solution

Provide a way to specify that we actually want all the cameras, including the logical cameras.
Provide a separate method to call to access logical cameras only.

Since AVCaptureDevice has the following properties:

isVirtualDevice: "A Boolean value that indicates whether the device consists of two or more physical devices." constituentDevices: "An array of physical devices that make up a virtual device."

On the Flutter side, the CameraDescription should take advantage of those properties so that users of the camera package can select hardware cameras, as well as logical cameras, and be able to pick the necessary one(s).

Proposal

This feature can be implemented as part of the camera package and is relatively simple to implement. Set the discoveryDevices to include the missing built-in logical cameras:

File: camera_avfoundation/ios/Classes/CameraPlugin.m Function: handleMethodCallAsync

NSMutableArray *discoveryDevices =
        [@[ AVCaptureDeviceTypeBuiltInWideAngleCamera, AVCaptureDeviceTypeBuiltInTelephotoCamera,
          AVCaptureDeviceTypeBuiltInDualCamera,
          AVCaptureDeviceTypeBuiltInDualWideCamera,
          AVCaptureDeviceTypeBuiltInTripleCamera ]
            mutableCopy];

Output - before:

[CameraStateBloc._updateDeviceCameraList] Found 2 device camera(s) for 'CameraLensDirection.back'.
[CameraStateBloc._updateDeviceCameraList]   com.apple.avfoundation.avcapturedevice.built-in_video:0
[CameraStateBloc._updateDeviceCameraList]   com.apple.avfoundation.avcapturedevice.built-in_video:5

Output - after:

[CameraStateBloc._updateDeviceCameraList] Found 3 device camera(s) for 'CameraLensDirection.back'.
[CameraStateBloc._updateDeviceCameraList]   com.apple.avfoundation.avcapturedevice.built-in_video:0
[CameraStateBloc._updateDeviceCameraList]   com.apple.avfoundation.avcapturedevice.built-in_video:6
[CameraStateBloc._updateDeviceCameraList]   com.apple.avfoundation.avcapturedevice.built-in_video:5

Result:

The newly returned logical camera properly switches hardware cameras when I zoom in/out as expected.

0x3dev commented 11 months ago

The AVCaptureDeviceTypeBuiltInTripleCamera is especially useful to let the system automatically manage the camera to help focus on large sensor camera devices, such as iPhone 14 Pro 48MP and newer. Related

FourwingsY commented 11 months ago

There is logical cameras in Android also.

flutter / flutter