search

gev2002 / react-native-vision-camera-barcodes-scanner

A plugin to scan barcodes using ML Kit Barcode Scanning
https://www.npmjs.com/package/react-native-vision-camera-barcodes-scanner
MIT License
5 stars 2 forks source link

[iOS] Broken with orientation update #4

Closed zzz08900 closed 1 week ago

zzz08900 commented 1 month ago

In mid June RNVC released initial support for orientation and the most recent release 4.5.0 had most of its orientation bugs iron out.

One major change (though @mrousavy does not consider it breaking) is RNVC no longer rotates (pixel displacing) the CMSampleBuffer on iOS and instead we need to handle orientation related stuff using Frame.orientation

Things all look great at first glance but after some real testing, I believe I had found a (long-existing) bug of MLKit barcode scanner on iOS: When generating VisionImage from CMSampleBuffer orientation specified in VisionImage.orientation is completely ignored and if the CMSampleBuffer is rotated then so will the detection results.

Considering the fact that Google does not really care about MLKit on iOS, I figure we need to come up with a solution on our own.

I'm proposing two three possible approaches:

  1. We do the rotation - allocate another CMSampleBuffer with proper size and copy the pixel over, exactly what RNVC was doing before orientation update. Feels like a big overkill.
  2. We rotate the detection results based on Frame.orientation, before sending them back to JS side. Could look pretty ugly.
  3. Add a section in the docs saying that Google messed up again and the detection results could be rotated so users should rotate it back. AKA, heads in the sand.

What do you guys think?

zzz08900 commented 1 month ago

I just opened a PR for approach 2. Be sure to test it and proof read it, like really carefully, before merging.

https://github.com/gev2002/react-native-vision-camera-barcodes-scanner/pull/5

mehtaanirudh commented 1 month ago

I agree with approach #2. I am trying to draw a rectangle on a QR code and it works fine on iOS but on Android the coordinates are always rotated. It would be good to return coodinates that are true to the frame's orientation

gev2002 commented 1 week ago

Hi, this will be fixed soon in the new release.