mrousavy / react-native-vision-camera

📸 A powerful, high-performance React Native Camera library.
https://react-native-vision-camera.com
MIT License
7.27k stars 1.06k forks source link

🐛 Camera stretched on Android #2142

Closed NuraQ closed 7 months ago

NuraQ commented 10 months ago

What's happening?

Camera is stretched on Android, behavior is not consistent as sometimes when closing the navbar and reopening the stretch is gone. image.

issue reproduced on these two devices:

Reproduceable Code

const device = useCameraDevice('back');    
const format =  getCameraFormat(device, [
        { resizeTo: { width: camContainerPosition?.width ?? '', height: 
        camContainerPosition?.height ?? '' } },
        { photoResolution: 'max' },
        { videoResolution: 'max' },
        { autoFocusSystem: 'contrast-detection' },
        { videoStabilizationModes: ['on', 'on', 'cinematic-extended'] } 
    ]);
    const fps = format.maxFps >= 240 ? 240 : format.maxFps;
    const touchToFocus = async (event) => {
        if (!device.supportsFocus || !isCameraFocusReady.current) return;
        let point = {
            x: Math.round(event.pageX - camLocation.x),
            y: Math.round(event.pageY - camLocation.y)
        };
        try {
            isCameraFocusReady.current = false;
            await cameraRef?.current?.focus(point);
            setTimeout(() => {
                isCameraFocusReady.current = true;
            }, 200);
        } catch (error) {

        }
    };
 <Camera
   format={format}
   isActive={isAndroid ? !paused : true}
   photo={true}
   orientation={'portrait'}
   ref={cameraRef}
   enableHighQualityPhotos={true}
   fps={fps}
   preset='photo'
   enableZoomGesture={true}
   onLayout={handleCameraLayout}
   device={device}
   zoom={device.neutralZoom}
   style={{ flex: 1 }}
   onTouchEnd={(x) => touchToFocus(x.nativeEvent)}
   />

Relevant log output

no logs

Camera Device

{"sensorOrientation":
"landscape-right",
"hardwareLevel": "full",
"maxZoom": 8,
"minZoom":1,
"supportsLowLightBoost":false,
"neutralZoom":1,
"physicalDevices":["wide-angle-camera"],
"supportsFocus":true,
"supportsRawCapture" true,
"isMultiCam":false,
"name":"BACK(0)",
"hasFlash" true,
"has Torch" true,
"position":"back",
"id":"0"}

Device

OnePlus 5T, Android version 10

VisionCamera Version

3.6.4

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

rsnr commented 10 months ago

@mrousavy Sponsoring this one as well :)

bglgwyng commented 10 months ago

Could you check if format is undefined?

mrousavy commented 10 months ago

@rsnr thank you!! ❤️

behavior is not consistent as sometimes when closing the navbar and reopening the stretch is gone.

@NuraQ so if you minimize the app and open it again, it looks correct? Then it's definitely a race condition in PreviewView's set size funcs.

I'll investigate this soon!

NuraQ commented 10 months ago

@mrousavy After testing on two devices, this was the result:

  1. on One Plus, it was consistent. "Minimizing" by the right button, or sending to background by the middle button, did not have impact (still camera is stretched).

  2. On Samsung, I think the back middle button doesn't always behave the same in terms of impact no stretching ( it some times fixes stretch issue and sometimes it doesn't).

mrousavy commented 10 months ago

okok lemme take a look over the weekend or next week, its a race condition

j-jonathan commented 9 months ago

Hey,

I've just started integrating react-native-vision-camera v3 into my project (i'm currently using react-native-camera) and encountered some issues related to aspect ratio on Android (similar to the problem described in this opened issue).

Usage

I am using the resizeMode='contain' mode to display the entire preview:

<Camera
    ref={this.cameraRef}
    style={{ position: 'absolute', top: 0, left: 0, right: 0, bottom: 0 }}
    device={device}
    format={format}
    isActive={true}
    photo={isPhoto}
    video={isVideo}
    audio={isVideo}
    resizeMode='contain'
/>

Tested on:

Issues

Observed Issues when I switch from one format to another:

For the first issue, it appears to be a problem related to react-native. I found a solution on this GitHub issue: https://github.com/facebook/react-native/issues/17968

Here is the patch to fix the issue:

diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
index 0aa8932..7b1c91e 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
@@ -242,4 +242,17 @@ class CameraView(context: Context) :
   override fun onCodeScanned(codes: List<Barcode>, scannerFrame: CodeScannerFrame) {
     invokeOnCodeScanned(codes, scannerFrame)
   }
+
+  override fun requestLayout() {
+    super.requestLayout()
+    post(measureAndLayout)
+  }
+
+  private val measureAndLayout = Runnable {
+    measure(
+        MeasureSpec.makeMeasureSpec(width, MeasureSpec.EXACTLY),
+        MeasureSpec.makeMeasureSpec(height, MeasureSpec.EXACTLY)
+    )
+    layout(left, top, right, bottom)
+  }
 }

For the second issue, it occurs when the surface is updated ("PreviewView Surface updated!"). The ratio is not necessarily updated, and I don't know why.

The only solution I found is to restart the entire device configuration.

Here is the patch to fix the issue:

diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
index 8ad60cc..15b85ff 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
@@ -187,6 +187,19 @@ class CameraSession(private val context: Context, private val cameraManager: Cam

         override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
           Log.i(TAG, "PreviewView Surface updated! ${holder.surface} $width x $height")
+
+          launch {
+            mutex.withLock {
+              try {
+                configuration?.let { configuration ->
+                  configureOutputs(configuration)
+                  configureCaptureRequest(configuration)
+                }
+              } catch (error: Throwable) {
+                Log.e(TAG, "Failed to configure CameraSession! Error: ${error.message}", error)
+              }
+            }
+          }
         }

         override fun surfaceDestroyed(holder: SurfaceHolder) {

However, this is not perfect, and the camera does not load instantly (refer to the 3rd video where you can see the incorrect ratio, and then 1 or 2 seconds later, the correct ratio after reloading the camera configuration).

@mrousavy > Do you have any idea where this issue might be coming from? I've tried various approaches but haven't found an alternative fix. I'm available to experiment with other solutions if needed.

Videos

To illustrate, here are videos of my application at different stages of the fixes:

Without any fix: https://github.com/mrousavy/react-native-vision-camera/assets/78229184/98129bb6-8a15-4ff4-a79a-e0d8b30ceb7d

With the first fix: https://github.com/mrousavy/react-native-vision-camera/assets/78229184/0d2e47d8-04ed-4c4e-b1c8-31895babd88e

With the two fixes: https://github.com/mrousavy/react-native-vision-camera/assets/78229184/6458f9e9-b6b6-4273-a4b5-58ea4ec72dc8

mrousavy commented 9 months ago

Hey - thanks for the insights man! I need to take a look at this, probably just a small fix to avoid any race conditions here. Will have some free time soon, maybe in 1-2 weeks

acesetmatch commented 8 months ago

Has there been any progress on this? Still experiencing the issue

bpeck81 commented 8 months ago

Not sure if this helps, but when displaying an image taken with the camera, I found that I had to swap the height and width properties returned from the photo object.

const photo = await cameraRef.current.takePhoto()

JuMaruhn commented 8 months ago

Here is a (dirty) quickfix that helps me:

Add the line:

private val aspectRatio: Float get() = size.width.toFloat() / size.height.toFloat()

to the class PreviewView in react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt.

Then change the methode onMeasure in the same file to:

@SuppressLint("DrawAllocation")
override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
  super.onMeasure(widthMeasureSpec, heightMeasureSpec)
  val viewWidth = MeasureSpec.getSize(widthMeasureSpec)
  val viewHeight = MeasureSpec.getSize(heightMeasureSpec)

  Log.i(TAG, "PreviewView onMeasure($viewWidth, $viewHeight)")

  val newWidth: Int
  val newHeight: Int

  val actualRatio = if (viewWidth > viewHeight) aspectRatio else 1f / aspectRatio
  if (viewWidth < viewHeight * actualRatio) {
    newHeight = viewHeight
    newWidth = (viewHeight * actualRatio).roundToInt()
  } else {
    newWidth = viewWidth
    newHeight = (viewWidth / actualRatio).roundToInt()
  }

  Log.d(TAG, "Measured dimensions set: $newWidth x $newHeight")
  setMeasuredDimension(newWidth, newHeight)
}
Alexis200007 commented 8 months ago

Here is a (dirty) quickfix that helps me:

Add the line:

private val aspectRatio: Float get() = size.width.toFloat() / size.height.toFloat()

to the class PreviewView in react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt.

Then change the methode onMeasure in the same file to:

@SuppressLint("DrawAllocation")
override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
  super.onMeasure(widthMeasureSpec, heightMeasureSpec)
  val viewWidth = MeasureSpec.getSize(widthMeasureSpec)
  val viewHeight = MeasureSpec.getSize(heightMeasureSpec)

  Log.i(TAG, "PreviewView onMeasure($viewWidth, $viewHeight)")

  val newWidth: Int
  val newHeight: Int

  val actualRatio = if (viewWidth > viewHeight) aspectRatio else 1f / aspectRatio
  if (viewWidth < viewHeight * actualRatio) {
    newHeight = viewHeight
    newWidth = (viewHeight * actualRatio).roundToInt()
  } else {
    newWidth = viewWidth
    newHeight = (viewWidth / actualRatio).roundToInt()
  }

  Log.d(TAG, "Measured dimensions set: $newWidth x $newHeight")
  setMeasuredDimension(newWidth, newHeight)
}

Thanks, it works for me

canhtvee commented 8 months ago

It's a race condition, I fixed it on JS side by init the camera dimension at zero, settimeout waiting for the camera launch, and then resizing the camera view.

Jigar2311 commented 8 months ago

i am also have this issue but when i make orientation lock to landscape

Alexis200007 commented 8 months ago

@Jigar2311 I had the same problem in landscape mode and the fix of @JuMaruhn fixed my problem.

iamtommcc commented 8 months ago

@JuMaruhn's fix partially fixed my issue - it made the preview properly fill its container (I was getting blank bars on the sides), but for me the preview was still distorted. This was on a Samsung Galaxy S22.

I too experienced some inconsistency, sometimes it would fix itself by flipping between front/back, minimising app, etc.

I was able to workaround this issue by not feeding in format straight away, and instead doing this:

function CameraPreview() {
  const [isInitialised, setIsInitialised] = useState(false);

  return (
    <View onLayout={() => setIsInitialised(true)}>
      <CameraView
        format={
          !isInitialised && Platform.OS === "android" ? undefined : format
        }
        resizeMode={"contain"}
        style={{
          aspectRatio: 3 / 4,
        }}
        {...yourOtherProps}
      />
    </View>
  );
}

This is very similar to @j-jonathan's approach except I think it's a little lighter/quicker than reinitialising the entire camera config.

Jigar2311 commented 8 months ago

@Jigar2311 I had the same problem in landscape mode and the fix of @JuMaruhn fixed my problem.

Sure @Alexis200007, let me see if it works for me

Jigar2311 commented 8 months ago

@Alexis200007 the stretching problem is fixes but i am using custom hight and width of video recording so i am passing the height and width in the format but it's not working for landscape.

const getPictureSizeAndroid = () => {
    let vQuality = statusSlice.currentVideoQuality;
    console.log('vQuality', vQuality);
    if (pDetail) {
        vQuality = pDetail?.videoHeightWidth?.width;
    }
    let returnValue = {};
    if (vQuality == staticStrings.VIDEO_QUALITY_1080) returnValue = { width: 1080, height: 1920 };
    else if (vQuality == staticStrings.VIDEO_QUALITY_720) returnValue = { width: 720, height: 1280 };
    else returnValue = { width: 1080, height: 1920 };
    return returnValue;
};

const format = useCameraFormat(device, [
    { fps: targetFps },
    {
        videoResolution: {
            height: isLandscape ? getPictureSizeAndroid()?.width : getPictureSizeAndroid()?.height,
            width: isLandscape ? getPictureSizeAndroid()?.height : getPictureSizeAndroid()?.width,
        },
    },
    { photoAspectRatio: screenAspectRatio },
    { photoResolution: 'max' },
]);
mgvictor7 commented 8 months ago

I have the same problem in the version 3.7.0 In version 3.5.1 there was no this problem

I have tried to solve the problem by updating the size in the camera style 250 milliseconds after the component has been mounted but I don't like this

...
const [adjustCameraSize, setAdjustCameraSize] = useState(false);

useEffect(() => {
    if (device && isFocussed && cameraPermission === Permissions.RESULTS.GRANTED) {
      setTimeout(() => {
        setAdjustCameraSize(true);
      }, 250);
    }
  }, [isFocussed, device, cameraPermission]);

...

<Camera
       style={[{ ...StyleSheet.absoluteFill }, { width: adjustCameraSize ? '100%' : '120%' }]}
       device={device}
        isActive
        codeScanner={codeScanner}
      />
mrousavy commented 8 months ago

Hey! Does this fix your issue maybe? https://github.com/mrousavy/react-native-vision-camera/pull/2377 (still testing)

mrousavy commented 8 months ago

Hey!

After 8 hours of debugging, I finally found the culprit! I fixed the preview stretching issue in this PR: https://github.com/mrousavy/react-native-vision-camera/pull/2377

If you appreciate my dedication and free support, please consider 💖 sponsoring me on GitHub 💖 so I can keep providing fixes and building out new features for my open-source libraries in my free time.

ben-qiu commented 7 months ago

I am still experiencing the same with 3.7.1, the preview is stretched.

Z-Hayk commented 7 months ago

The same with 3.8.2, the preview is stretched.

hieupvXmasEve commented 7 months ago

@JuMaruhn's fix partially fixed my issue - it made the preview properly fill its container (I was getting blank bars on the sides), but for me the preview was still distorted. This was on a Samsung Galaxy S22.

I too experienced some inconsistency, sometimes it would fix itself by flipping between front/back, minimising app, etc.

I was able to workaround this issue by not feeding in format straight away, and instead doing this:

function CameraPreview() {
  const [isInitialised, setIsInitialised] = useState(false);

  return (
    <View onLayout={() => setIsInitialised(true)}>
      <CameraView
        format={
          !isInitialised && Platform.OS === "android" ? undefined : format
        }
        resizeMode={"contain"}
        style={{
          aspectRatio: 3 / 4,
        }}
        {...yourOtherProps}
      />
    </View>
  );
}

This is very similar to @j-jonathan's approach except I think it's a little lighter/quicker than reinitialising the entire camera config.

it works for me. Thanks

piwko28 commented 7 months ago

it works for me. Thanks

huntfpoly12 Works for me as well as a temp. fix, but not always.

chramos commented 7 months ago

in my case I'm not specifying the format prop and I'm still getting the issue

"react-native-vision-camera": "^3.8.2"

Edit: Downgrading to 3.6.4 the issue is gone

JshGrn commented 7 months ago

I am also not specifying the format prop and none of the fixes work for me on a Nokia G22. Whatever settings I use the height is slightly stretched, switching between camera and app doesn't fix either. My app is hardcoded to be portrait only.

grantsingleton commented 7 months ago

Still happening on 3.8.2. None of the temp fixes work reliably for me. I'm sponsoring this issue @mrousavy 🙌

metuuu commented 7 months ago

I'm having this issue also.

mrousavy commented 7 months ago

Thanks @grantsingleton! Will look into this soon!

JshGrn commented 7 months ago

Its definitely a race condition because whilst developing other areas of the app when I have 'completed' my work flow and passed back to the camera screen its fine. Weirdly, it actually flips between 3, stretched height, really stretched height (square becomes long rectangle) and normal.

boiboif commented 7 months ago

Same issue in landscape after upgrade from 2.x to 3.8.2.

krzychuuu132 commented 7 months ago

Same issue :/

Dingenis commented 7 months ago

We are also experiencing this issue. We enormously respect and love what you @mrousavy are doing so our small company became a sponsor. If you have any time this issue would have our preference.

mrousavy commented 7 months ago

Thanks @Dingenis, which company?

I'll take a look at this soon 🙏

Menardi commented 7 months ago

Hey @mrousavy I'm a relatively new sponsor, so I'm not certain how asking for issue prioritisation works. If it's just asking here in the comments, then I'd really appreciate if you could focus on this issue too. Thank you!

mrousavy commented 7 months ago

Thanks for the sponsorship @Menardi, I'll take a look soon! 🙏

JshGrn commented 7 months ago

I really don't want to sound ungrateful or rude here, but this issue is widespread and totally broken for all android users. Its a massive issue. I am not sure how you don't have it in your demo app. I also really feel like this is the only repo I have ever seen where in the issues are target sponsorship amounts.

Why not make this require a license where people have to pay for it instead if the issue is your time and needing to be funded? I completely agree that time is not free so I don't expect it for nothing.

The thing which stops me sponsoring is seeing your reply's say you will do it soon, what is soon, what am I sponsoring? Will it be this month, this year?

mrousavy commented 7 months ago

this issue is widespread and totally broken for all android users. Its a massive issue.

@JshGrn It is widespread, and I also encountered it a few times. I won't consider this "totally broken" or a "massive issue", the camera works, it just looks stretched in some cases (if you are not using the timeout workaround). I would consider something "totally broken"/a "massive issue" if the app crashes, or the camera stays completely black always.

I am not sure how you don't have it in your demo app.

I have it in the demo app, I can reproduce it.

I also really feel like this is the only repo I have ever seen where in the issues are target sponsorship amounts.

Yea, that's how things work here. I work on stuff that I love doing and innovate / build new features, and if someone who uses my library for their benefit encounters an issue with it, they can always pay me to fix some errors/edge cases.

You get new features for free (e.g. Frame Processors, buffer compression, video pipeline, etc), but if something does not work in your app, that's an edge case.

Why not make this require a license where people have to pay for it instead if the issue is your time and needing to be funded? I completely agree that time is not free so I don't expect it for nothing.

I'm not sure if you've worked on open-source projects with thousands of users and countless of posts/questions/issues each week but it is insanely time consuming. In the past few weeks, I was working full time on open-source. I thought about licensing, but then again this is hard to accomplish. Many people won't use libraries with unconventional licenses, some people won't pay, and it requires legal consulting which is always costly.

Also, it is technically more fair if someone requires additional assistance/time from me to fix something in this library to pay me more for that time than someone who can just use it without any issues.

As you say - time is not free, and especially when it comes to stuff like native modules, low-level APIs, and especially especially when it comes to Camera APIs as those are considered one of the most difficult APIs on phones. It requires C++ knowledge, Camera knowledge, even OpenGL/Graphics knowledge on Android. It's also pretty much impossible to debug 10MB NATIVE GPU buffers flowing in at 60 times a second. Not to mention the bridging part, JSI, TypeScript, etc. It's just hard to find assistance here, so right now I'm pretty much the only person developing features and fixes for VisionCamera, apart from a few contributors from time to time who send some small PRs for fixes (thankful for everyone of them!)

The thing which stops me sponsoring is seeing your reply's say you will do it soon, what is soon, what am I sponsoring? Will it be this month, this year?

Originally I set up GitHub sponsors so people could thank me for building VisionCamera. Without VisionCamera, many Camera apps would simply not be possible to build in RN today. It would require thousands of USD/EUR of effort to build this (probably as a native module in RN) if VisionCamera wasn't here, so my project saved a lot of projects thousands of dollars. I'm active in a ton of forums about AVFoundation and Camera2/CameraX APIs, have a lot of experience with Cameras now through VisionCamera, and spent more than 2.000 hours on this open-source project already.

Nevertheless, not a lot of people use sponsors or say thanks to authors/maintainers of open-source - I mean why should they, they can use it free and can save dollars for other projects in their business. It's just not how the world works, not everybody has money to give away, I understand that.

So now I extended the sponsors program so people can pay me for anything I wouldn't normally work on. For example, this preview stretching is an issue a lot of people are seeing, and if a lot of people sponsors some amounts, I can dedicate time to fix the issue - kinda like crowdfunding. Some people do $70, $100, some $200, etc. - if this takes 60 hours to complete, a competetive salary would require $9.000 - but since we're in open-source I'm doing it for way less than that, and that's fine.

what is soon, what am I sponsoring? Will it be this month, this year?

Sponsors are not consulting contracts though, so there is no deadline or promised delivery or something. That's also why it is much less than normal competetive salaries. If you pay me the full competetive salary (e.g. by contacting me through my consulting agency) you can get a deadline and I'll do it within a week.


Either way with all that said, I am working on this. A few people have sponsored this issue (thank you to my sponsors!!! ❤️), and I am working on the persistent capture session (https://github.com/mrousavy/react-native-vision-camera/pull/2494) which will be the foundation where the preview view is always synced. This is not something that can be fixed in 5 hours or so, this requires a ton of debugging, restructuring, understanding of native preview surfaces, etc. It's just hard and a lot of effort, otherwise I would've fixed it already (or someone could've submitted a PR already).

So in short; if you want this fixed faster, sponsor me on GitHub. If you don't sponsor me on GitHub or pay me for my work, that's also completely fine - I'm not saying you should - it's open-source, you can use my code however you want :)

Dingenis commented 7 months ago

Thanks @Dingenis, which company?

I'll take a look at this soon 🙏

Hey @mrousavy, that's great! Thank you! Our company is Spaces Experiences.

james-cohen commented 7 months ago

Unfortunately this is still occurring even on the new beta release, albeit differently. Before the preview was stretched horizontally, but it now stretches vertically, then on refreshing the camera it stretches horizontally.

Interestingly, if I remove the videoAspectRatio prop from the useCameraFormat hook, it still stretches vertically on initialisation, but on refresh it is then normal.

mrousavy commented 7 months ago

Created a PR to fix this issue once and for all: https://github.com/mrousavy/react-native-vision-camera/pull/2519 💪🚀 Thanks to everyone who sponsored me on GitHub to fix this bug!! 💖

Try the PR and see if it works (by checking out that branch and running the example app)

Menardi commented 7 months ago

Thanks for your hard work @mrousavy! I've tested this fix in the example app on a Pixel 7a (Android 14), and a Pixel 3a (Android 12) and the view is no longer stretched when using the fix/preview-stretching branch. 🙌

I did notice one unexpected side effect on the 7a, which is that the default zoom level seems to be much more zoomed in on the fixed branch. I didn't see this behaviour on the 3a, so I presume it must be something to do with the 7a having two rear cameras. Setting the prop zoom={device.neutralZoom} fixed this so the zoom level was consistent between main and fix/preview-stretching.

I don't think it's a bug, but thought it was worth mentioning as it might come up once the fix is released. I've included some photos taken on each branch below if you're interested.

Either way, thanks very much for the fix!

Screenshots showing default zoom differences (Pixel 7a) | Default zoom: main | Default zoom: fix/preview-stretching | |-----|---| | | |
mrousavy commented 7 months ago

Thanks for your hard work @mrousavy! I've tested this fix in the example app on a Pixel 7a (Android 14), and a Pixel 3a (Android 12) and the view is no longer stretched when using the fix/preview-stretching branch. 🙌

Great to hear!!! 💪

I did notice one unexpected side effect on the 7a, which is that the default zoom level seems to be much more zoomed in on the fixed branch. I didn't see this behaviour on the 3a, so I presume it must be something to do with the 7a having two rear cameras. Setting the prop zoom={device.neutralZoom} fixed this so the zoom level was consistent between main and fix/preview-stretching.

Huh, interesting. Maybe this has something to do with me not resetting the zoom on device change properly, but it is weird nevertheless... 🤔

correafederico25 commented 7 months ago

Hello! I added version 3.9.0-beta.2 and now I am experiencing a different phenomenon. Before, the problem was that the camera preview stretched vertically, now the problem is that it stretches horizontally.

Photo from React native vision camera

Screenshot_20240208-123014

Photo in normal conditions

Screenshot_20240208-123024

Any suggestions or recommendations you can give me so this will work on Android? Here is my component.

    <Camera
      ref={cameraRef}
      style={StyleSheet.absoluteFill}
      device={device}
      isActive={isActive}
      photo={true}
    />
luicfrr commented 6 months ago

Hello! I added version 3.9.0-beta.2 and now I am experiencing a different phenomenon. Before, the problem was that the camera preview stretched vertically, now the problem is that it stretches horizontally.

@mrousavy I'm currently with this exactly same issue but looking into 3470b4f I can see that you inverted some width/height props, reverting these props to correct width/height order all stretching issues are gone.

Here's a patch I made:

diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
index fef6998..c339ea7 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
@@ -313,7 +313,7 @@ class CameraSession(private val context: Context, private val cameraManager: Cam
       )
       outputs.add(output)
       // Size is usually landscape, so we flip it here
-      previewView?.setSurfaceSize(size.width, size.height)
+      previewView?.setSurfaceSize(size.height, size.width)
     }

     // CodeScanner Output
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
index c862be7..2d26fe9 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
@@ -43,13 +43,13 @@ class PreviewView(context: Context, callback: SurfaceHolder.Callback) : SurfaceV
     holder.addCallback(callback)
   }

-  suspend fun setSurfaceSize(width: Int, height: Int) {
+  suspend fun setSurfaceSize(height: Int, width: Int) {
     withContext(Dispatchers.Main) {
-      size = Size(width, height)
+      size = Size(height, width)
       Log.i(TAG, "Setting PreviewView Surface Size to $size...")
       requestLayout()
       invalidate()
-      holder.resize(width, height)
+      holder.resize(height, width)
     } 
   }

@@ -73,7 +73,7 @@ class PreviewView(context: Context, callback: SurfaceHolder.Callback) : SurfaceV

   private fun getSize(contentSize: Size, containerSize: Size, resizeMode: ResizeMode): Size {
     // TODO: Take sensor orientation into account here
-    val contentAspectRatio = contentSize.height.toDouble() / contentSize.width
+    val contentAspectRatio = contentSize.width.toDouble() / contentSize.height
     val containerAspectRatio = containerSize.width.toDouble() / containerSize.height

     val widthOverHeight = when (resizeMode) {
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/CameraCharacteristics+getPreviewSize.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/CameraCharacteristics+getPreviewSize.kt
index ecd525c..e758c45 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/CameraCharacteristics+getPreviewSize.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/CameraCharacteristics+getPreviewSize.kt
@@ -9,7 +9,7 @@ fun getMaximumPreviewSize(): Size {
   // See https://developer.android.com/reference/android/hardware/camera2/params/StreamConfigurationMap
   // According to the Android Developer documentation, PREVIEW streams can have a resolution
   // of up to the phone's display's resolution, with a maximum of 1920x1080.
-  val display1080p = Size(1920, 1080)
+  val display1080p = Size(1080, 1920)
   val displaySize = Size(
     Resources.getSystem().displayMetrics.widthPixels,
     Resources.getSystem().displayMetrics.heightPixels
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/SurfaceHolder+resize.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/SurfaceHolder+resize.kt
index cc0fe9c..6ef3880 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/SurfaceHolder+resize.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/SurfaceHolder+resize.kt
@@ -7,7 +7,7 @@ import kotlin.coroutines.resume
 import kotlinx.coroutines.suspendCancellableCoroutine

 @UiThread
-suspend fun SurfaceHolder.resize(width: Int, height: Int) {
+suspend fun SurfaceHolder.resize(height: Int, width: Int) {
   return suspendCancellableCoroutine { continuation ->
     val currentSize = this.surfaceFrame
     if (currentSize.width() == width && currentSize.height() == height) {

This fixed the issue for me

snips11 commented 6 months ago

I have just tried the beta version and seeing the horizontal stretch.

mrousavy commented 6 months ago

Okay yea then I'm guessing this is related to the Orientation issue (https://github.com/mrousavy/react-native-vision-camera/issues/1891), because the devices you are testing on have a different Camera Hardware-Sensor orientation than my devices. We need to take that into account.

inzqne commented 6 months ago

Okay yea then I'm guessing this is related to the Orientation issue (#1891), because the devices you are testing on have a different Camera Hardware-Sensor orientation than my devices. We need to take that into account.

@mrousavy How come all of the stretching issues (horizontal and vertical) are not present in 3.6.4? Were there any features or changes that altered the resizing methods after 3.6.4?

JshGrn commented 6 months ago

I had these issues on 3.6.4 so it wasn't introduced there, I think its a mixture of things for multiple devices.

Interestingly though, this is now for me broken in a way that I can work around 100% of the time. The initial camera loads with the width and height inverted. The 'fix' is to set them to 0 and activate the camera, and then in a useEffect set them to the devices width and height again. This loads the camera correctly, although is lower quality to the native camera app.

mrousavy commented 6 months ago

Yea the issue always existed, it is a race condition which apparently got worse after I made the codebase more predictable with the "persistent capture session" as this was the only part that was not predictable.

I spent quite a lot of hours trying to fix this, and thought I fixed it a couple of times since I couldn't reproduce it on my devices anymore, but then had to switch back to other stuff and my main job, as well as the crashing issues on Samsung were higher priority than that.

This will definitely be fixed in V4, maybe I can also find a quick solution to fix it in V3 now while V4 is still brewing