🐛 Front camera recording is inverted when going from Back to Front camera during recording on Android

likecarson commented 4 months ago

What's happening?

On an Android device, while taking a video recording with the Back camera started, if user switches to Front camera during the same recording the Front video recording becomes inverted -- it looks okay during the recording session but the saved video recording of the Front camera is inverted (upside-down).

Interestingly, if I start the recording using the Front camera then switch to Back camera the recording of the Front video is fine.

Reproduceable Code

const [cameraPosition, setCameraPosition] = useState<'front' | 'back'>('back');
  const cameraDevice = useCameraDevice(cameraPosition);

<Camera
            ref={cameraRef}
            device={cameraDevice}
            torchEnabled={torchEnabled}
            onRotate={useCallback(() => {
              setCameraPosition((p) => (p === 'back' ? 'front' : 'back'));
            }, [])}
            onTorchToggle={() => setTorchEnabled((prev) => !prev)}
          />

Relevant log output

Back Camera:
{"formats": [{"autoFocusSystem": "none", "fieldOfView": 115.98923300825525, "maxFps": 30, "maxISO": 1600, "minFps": 2, "minISO": 25, "photoHeight": 144, "photoWidth": 176, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "none", "fieldOfView": 115.98923300825525, "maxFps": 30, "maxISO": 1600, "minFps": 2, "minISO": 25, "photoHeight": 240, "photoWidth": 320, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "none", "fieldOfView": 115.98923300825525, "maxFps": 30, "maxISO": 1600, "minFps": 2, "minISO": 25, "photoHeight": 288, "photoWidth": 352, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "none", "fieldOfView": 115.98923300825525, "maxFps": 30, "maxISO": 1600, "minFps": 2, "minISO": 25, "photoHeight": 480, "photoWidth": 640, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "none", "fieldOfView": 115.98923300825525, "maxFps": 30, "maxISO": 1600, "minFps": 2, "minISO": 25, "photoHeight": 720, "photoWidth": 1280, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "none", "fieldOfView": 115.98923300825525, "maxFps": 30, "maxISO": 1600, "minFps": 2, "minISO": 25, "photoHeight": 960, "photoWidth": 1280, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}], "hardwareLevel": "limited", "hasFlash": false, "hasTorch": false, "id": "10", "isMultiCam": false, "maxExposure": 6, "maxZoom": 1, "minExposure": -6, "minFocusDistance": 999.999985098839, "minZoom": 1, "name": "10 (BACK) androidx.camera.camera2", "neutralZoom": 1, "physicalDevices": ["ultra-wide-angle-camera"], "position": "back", "sensorOrientation": "landscape-left", "supportsFocus": false, "supportsLowLightBoost": false, "supportsRawCapture": false}

Front Camera:
{"formats": [{"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 1440, "photoWidth": 1920, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 1080, "photoWidth": 1920, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 960, "photoWidth": 1920, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 1200, "photoWidth": 1600, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 1080, "photoWidth": 1440, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 960, "photoWidth": 1280, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 720, "photoWidth": 1280, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 768, "photoWidth": 1024, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 600, "photoWidth": 800, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 480, "photoWidth": 720, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 480, "photoWidth": 640, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 360, "photoWidth": 640, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 288, "photoWidth": 352, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}, {"autoFocusSystem": "contrast-detection", "fieldOfView": 77.70146040253039, "maxFps": 30, "maxISO": 1000, "minFps": 7, "minISO": 100, "photoHeight": 240, "photoWidth": 320, "supportsDepthCapture": false, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 720, "videoStabilizationModes": [Array], "videoWidth": 1280}], "hardwareLevel": "full", "hasFlash": true, "hasTorch": true, "id": "1", "isMultiCam": true, "maxExposure": 24, "maxZoom": 12.931958198547363, "minExposure": -24, "minFocusDistance": 9.800000093460085, "minZoom": 1, "name": "1 (FRONT) androidx.camera.camera2", "neutralZoom": 1, "physicalDevices": ["wide-angle-camera", "wide-angle-camera", "telephoto-camera"], "position": "front", "sensorOrientation": "landscape-right", "supportsFocus": true, "supportsLowLightBoost": false, "supportsRawCapture": false}

Camera Device

Back Camera
{
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "limited",
  "maxZoom": 1,
  "minZoom": 1,
  "maxExposure": 6,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "ultra-wide-angle-camera"
  ],
  "supportsFocus": false,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 999.999985098839,
  "minExposure": -6,
  "name": "10 (BACK) androidx.camera.camera2",
  "hasFlash": false,
  "hasTorch": false,
  "position": "back",
  "id": "10"
}

Front Camera
{
  "formats": [],
  "sensorOrientation": "landscape-right",
  "hardwareLevel": "full",
  "maxZoom": 12.931958198547363,
  "minZoom": 1,
  "maxExposure": 24,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera",
    "wide-angle-camera",
    "telephoto-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": true,
  "minFocusDistance": 9.800000093460085,
  "minExposure": -24,
  "name": "1 (FRONT) androidx.camera.camera2",
  "hasFlash": true,
  "hasTorch": true,
  "position": "front",
  "id": "1"
}

Device

Android Pixel 6 Pro API 34 (emulator), Pixel 5 API 34 (physical device)

VisionCamera Version

4.0.3

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

[ ] I am using Expo
[ ] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

mrousavy commented 4 months ago

What does inverted mean? Please attach a sample video.

likecarson commented 4 months ago

Here is a screen recording using the Android Pixel 6 Pro API 34 emulator. The same happens on physical device:

mrousavy commented 4 months ago

Okay, this is a CameraX bug then, nothing I am doing explicitly - could you create a bug report in the Google CameraX issue tracker about this? Thanks!

likecarson commented 4 months ago

@mrousavy I think I figured out the issue -- at least doing this change fixes it for me:

In this file: node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt

I commented out line 308: val video = VideoCapture.Builder(recorder).also { video -> // Configure Video Output // video.setMirrorMode(MirrorMode.MIRROR_MODE_ON_FRONT_ONLY)

And the issue was resolved. I think this is because in my use case, the recording starts using the back camera and the CameraX must set a certain orientation, and when I switch to Front camera during the same recording the orientation is not updated to relative to Front camera, so mirrorMode causes the inversion.

I'm not sure if there is a way to adjust the code to check if the orientation is correct before applying the setMirrorMode or maybe this is the desired behavior just an issue with CameraX as you say.

Thanks.

mrousavy commented 2 months ago

Hey - I think I fixed this issue in the latest releases. I update orientation immediately after switching devices. Can you confirm if that works for you?

likecarson commented 2 months ago

Hi Marc,

Yes the latest version seems to be working now! No more inverted selfie videos on Android.

I appreciate your follow up and all the work you are doing for this great package.

Thanks, Carson

On Wed, Jul 10, 2024 at 8:17 AM Marc Rousavy @.***> wrote:

Hey - I think I fixed this issue in the latest releases. I update orientation immediately after switching devices. Can you confirm if that works for you?

— Reply to this email directly, view it on GitHub https://github.com/mrousavy/react-native-vision-camera/issues/2854#issuecomment-2220360780, or unsubscribe https://github.com/notifications/unsubscribe-auth/BDDN2OQZNIOHYOC43ADQHIDZLUQ6LAVCNFSM6AAAAABHLQAYRCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRQGM3DANZYGA . You are receiving this because you authored the thread.Message ID: @.***>

mrousavy commented 2 months ago

Awesome thank you. If you appreciate my work and dedication please consider sponsoring me on GitHub :)

mrousavy / react-native-vision-camera