🐛 Square ratio preview has low quality

ppanwin10 commented 3 weeks ago

What's happening?

Hi,

I am not sure if I miss anything, I have the following setup for the camera format:

 const screenAspectRatio = 1 / 1
 const format = useCameraFormat(device, [
    { fps: targetFps },
    { videoAspectRatio: screenAspectRatio },
    { videoResolution: 'max' },
    { photoAspectRatio: screenAspectRatio },
    { photoResolution: 'max' },
  ])

ResizeMode = "contain"

The preview quality for all kinds of ratios (e.g. 16:9, 4:3) are high enough, except for 1:1.

https://github.com/mrousavy/react-native-vision-camera/assets/77748079/2df3e901-b08b-4ffe-98c2-ffdd4d598768

Reproduceable Code

<ReanimatedCamera
                style={StyleSheet.absoluteFill}
                device={device}
                isActive={isActive}
                ref={camera}
                onInitialized={onInitialized}
                onError={onError}
                onStarted={() => 'Camera started!'}
                onStopped={() => 'Camera stopped!'}
                format={format}
                fps={fps}
                photoHdr={photoHdr}
                videoHdr={videoHdr}
                photoQualityBalance="quality"
                lowLightBoost={device.supportsLowLightBoost && enableNightMode}
                enableZoomGesture={false}
                animatedProps={cameraAnimatedProps}
                exposure={0}
                enableFpsGraph={true}
                orientation="portrait"
                photo={true}
                video={true}
                audio={microphone.hasPermission}
                enableLocation={location.hasPermission}
                frameProcessor={frameProcessor}
                resizeMode={'contain'}
              />

Relevant log output

11:23:19.460: [info] 📸 VisionCamera.didSetProps(_:): Updating 34 props: [lowLightBoost, onAverageFpsChanged, exposure, photoHdr, onShutter, resizeMode, onStarted, preview, onStopped, collapsable, onCodeScanned, left, zoom, photo, onViewReady, orientation, enableFrameProcessor, enableLocation, position, isActive, top, enableZoomGesture, photoQualityBalance, format, audio, right, video, cameraId, fps, onError, bottom, videoHdr, onInitialized, enableBufferCompression]
11:23:19.464: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
11:23:19.467: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: true, outputsChanged: true, videoStabilizationChanged: true, orientationChanged: true, formatChanged: true, sidePropsChanged: true, torchChanged: true, zoomChanged: true, exposureChanged: true, audioSessionChanged: true, locationChanged: true)
11:23:19.467: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Input Device...
11:23:19.467: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Camera com.apple.avfoundation.avcapturedevice.built-in_video:0...
11:23:19.481: [info] 📸 VisionCamera.configureDevice(configuration:): Successfully configured Input Device!
11:23:19.481: [info] 📸 VisionCamera.configureOutputs(configuration:): Configuring Outputs...
11:23:19.481: [info] 📸 VisionCamera.configureOutputs(configuration:): Adding Photo output...
11:23:19.485: [info] 📸 VisionCamera.configureOutputs(configuration:): Adding Video Data output...
11:23:19.487: [info] 📸 VisionCamera.configureOutputs(configuration:): Successfully configured all outputs!
11:23:19.488: [info] 📸 VisionCamera.configureFormat(configuration:device:): Configuring Format (3696x3024 | 352x288@60.0 (ISO: 32.0..3072.0))...
11:23:19.488: [info] 📸 VisionCamera.configureFormat(configuration:device:): Successfully configured Format!
11:23:19.489: [info] 📸 VisionCamera.getPixelFormat(for:): Available Pixel Formats: ["420v", "420f", "BGRA"], finding best match... (pixelFormat="yuv", enableHdr={false}, enableBufferCompression={false})
11:23:19.489: [info] 📸 VisionCamera.getPixelFormat(for:): Using PixelFormat: 420f...
[I] <MMKV_OSX.cpp:99::setIsInBackground> g_isInBackground:0
[I] <libMMKV.mm:313::+[MMKV didBecomeActive]> isInBackground:0
11:23:19.774: [info] 📸 VisionCamera.onCameraStarted(): Camera started!
11:23:19.774: [info] 📸 VisionCamera.onSessionInitialized(): Camera initialized!
11:23:19.774: [info] 📸 VisionCamera.configure(_:): Beginning AudioSession configuration...
11:23:19.775: [info] 📸 VisionCamera.configureAudioSession(configuration:): Configuring Audio Session...
Camera initialized!
11:23:19.775: [info] 📸 VisionCamera.configureAudioSession(configuration:): Adding Audio input...

Camera Device

{
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "maxZoom": 16,
  "minFocusDistance": 12,
  "minZoom": 1,
  "hardwareLevel": "full",
  "position": "back",
  "hasTorch": true,
  "minExposure": -8,
  "hasFlash": true,
  "name": "Back Camera",
  "sensorOrientation": "landscape-right",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "formats": [],
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "neutralZoom": 1,
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false
}

Device

iPhont 11 IOS17.4.1

VisionCamera Version

4.0.3

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

[ ] I am using Expo
[X] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

ppanwin10 commented 3 weeks ago

BTW, I think this won't be a big problem if we go with cutting the view manually.

mrousavy commented 3 weeks ago

What resolution does format have?

ppanwin10 commented 2 weeks ago

What resolution does format have?

always "max"

mrousavy commented 2 weeks ago

No, I'm talking about the format. Not your format filter. Log the format that you get.

ppanwin10 commented 2 weeks ago

ok, here you go. {"autoFocusSystem": "phase-detection", "fieldOfView": 97.75704193115234, "maxFps": 60, "maxISO": 3072, "minFps": 1, "minISO": 32, "photoHeight": 3024, "photoWidth": 3696, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 288, "videoStabilizationModes": ["auto", "off"], "videoWidth": 352}

mrousavy commented 2 weeks ago

videoWidth: 352, videoHeight: 288 <-- that's why your format is so low quality.

The Preview view is also a video stream obviously, so it depends on the video size in the format. Choose a format with a higher video size. You could also just write your own format filter if you want.

mrousavy / react-native-vision-camera