🐛 Orientation in output not correct when camera facing up/down.

alexstanbury commented 1 month ago

What's happening?

First of all, thanks so much for implementing this feature!

My users often take photos of things on the floor or ceiling. Unfortunately the image is rotated incorrectly if the camera is in landscape orientation but facing up or down, it always output a portrait orientation image. It works correctly if the sensor is more than approx 20 degrees from being flat, but any less than that it doesn't.

I know this problem well because I had the same issue, which I haven't been able to fix, with my own orientation sensing code that I've been using up til now to correctly handle image rotation. I was hoping I could do away with it now this library supports orientation.

Example app on iOS shows the same behaviour - as well as the result that is displayed immediately after capture showing incorrectly if in landscape mode - I also had to lock orientation to portrait or else I got a semi transparent grey bar showing across the preview view.

Shadowlens on Android shows the same behaviour, but interestingly I can't take a landscape photo at all on the iOS (iPhone and iPad) version of it no matter what (enabling/disabling portrait lock makes no difference - is there a separate bug tracker for shadowlens? I also had an issue with not being able to delete a photo on Android).

The native camera apps work as expected.

Reproduceable Code

<Camera
            enableZoomGesture={true}
            photo={true}
            ref={cameraRef}
            style={styles.camera}
            device={device}
            isActive={isActive}
            onTouchEnd={focusCamera}
          />

Relevant log output

Logs not relevant
Logs not relevant

Camera Device

iPhone XR:

{
  "minFocusDistance": 12,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "minZoom": 1,
  "sensorOrientation": "portrait",
  "hasTorch": true,
  "isMultiCam": false,
  "maxZoom": 16,
  "position": "back",
  "neutralZoom": 1,
  "supportsFocus": true,
  "supportsRawCapture": false,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "hardwareLevel": "full",
  "supportsLowLightBoost": false,
  "maxExposure": 8,
  "hasFlash": true,
  "name": "Back Camera",
  "minExposure": -8,
  "formats": []
}

Device

Pixel 6 (Android 14), iPhone XR (iOS 17.0.3), iPad 2022 (iOS 17.5.1)

VisionCamera Version

4.5.1

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

[X] I am using Expo
[X] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

maintenance-hans[bot] commented 1 month ago

Guten Tag, Hans here. Dankeschön for ze detailed report! It seems you are experiencing a common issue with orientation handling when the camera is facing up or down.

Please ensure that you are using ze latest version of ze library, and consider trying to lock ze orientation in your app settings to see if that helps mitigate ze issue. Since you have also mentioned issues with ze Shadowlens app, it might be beneficial to check there as well.

If you find this project helpful, feel free to consider sponsoring mrousavy for faster assistance. 🥨

Note: If you think I made a mistake by closing this issue, please ping @mrousavy to take a look.

mrousavy commented 1 month ago

Actually this is a valid issue @maintenance-hans.

I think this is because .face-up and .face-down are just casted to portrait here:

https://github.com/mrousavy/react-native-vision-camera/blob/77e98178f84824a0b1c76785469413b64dc96046/package/ios/Core/Types/Orientation.swift#L69-L70

This is a bit tricky, since we technically shouldn't fire the onOrientationChanged event if it's up/down.... Maybe I can explicitly swallow that event, but it's a bit tricky and requires lots of testing.

alexstanbury commented 1 month ago

Yeah, it's a tricky problem.

I managed to fix my own orientation handler (that uses expo sensors) to an acceptable level by ignoring the sensor information when the device entered the nearly flat orientation zone and just using the orientation it was detected as before that point, then switching on the listener again when it left that zone.

I can still very occasionally get it to be incorrect but only if I do some ninja-like movements.

mrousavy / react-native-vision-camera