Frame positioning for IOS is messed up

What happened?

Turns out frame calculation of OCR for photos on IOS is messed up in @react-native-ml-kit/text-recognition.

Have found that the frame is misinterpreted and should be:

top: is misinterpreted as left height and width are switched. and left: would be screen width - their top - their height.

Version

@react-native-ml-kit/barcode-scanning: version@react-native-ml-kit/face-detection: version@react-native-ml-kit/identify-languages: version@react-native-ml-kit/image-labeling: version@react-native-ml-kit/text-recognition: version@react-native-ml-kit/translate-text: version

Which ML Kit packages do you use?

[ ] @react-native-ml-kit/barcode-scanning
[ ] @react-native-ml-kit/face-detection
[ ] @react-native-ml-kit/identify-languages
[ ] @react-native-ml-kit/image-labeling
[X] @react-native-ml-kit/text-recognition
[ ] @react-native-ml-kit/translate-text

What platforms are you seeing this issue on?

[ ] Android
[X] iOS

System Information

RN 0.73.6 System: OS: macOS 14.6.1 CPU: (8) arm64 Apple M3 Memory: 84.88 MB / 16.00 GB Shell: version: "5.9" path: /bin/zsh Binaries: Node: version: 18.20.0 path: ~/.nvm/versions/node/v18.20.0/bin/node Yarn: version: 1.22.22 path: /opt/homebrew/bin/yarn npm: version: 10.5.0 path: ~/.nvm/versions/node/v18.20.0/bin/npm Watchman: version: 2024.09.02.00 path: /opt/homebrew/bin/watchman Managers: CocoaPods: version: 1.15.2 path: /opt/homebrew/bin/pod SDKs: iOS SDK: Platforms:

DriverKit 24.0
iOS 18.0
macOS 15.0
tvOS 18.0
visionOS 2.0
watchOS 11.0 Android SDK: API Levels:
"28"
"31"
"33"
"34" Build Tools:
28.0.3
30.0.3
31.0.0
33.0.1
34.0.0 System Images:
android-34 | Google APIs ARM 64 v8a
android-34 | Google Play ARM 64 v8a Android NDK: 25.1.8937393 IDEs: Android Studio: 2024.1 AI-241.15989.150.2411.11948838 Xcode: version: 16.0/16A242d path: /usr/bin/xcodebuild Languages: Java: version: 17.0.12 path: /opt/homebrew/bin/javac Ruby: version: 3.3.5 path: /opt/homebrew/opt/ruby/bin/ruby npmPackages: "@react-native-community/cli": Not Found react: installed: 18.2.0 wanted: 18.2.0 react-native: installed: 0.73.6 wanted: 0.73.6 react-native-macos: Not Found npmGlobalPackages: "react-native": Not Found Android: hermesEnabled: true newArchEnabled: false iOS: hermesEnabled: true newArchEnabled: false

Steps to Reproduce

const {blocks} = await TextRecognition.recognize(`file://${photo.path}`);

expand lines then filter blocks somehow. (or don't)

use "react-native-fs"

to copy original image to somewhere on phone and save json encoded blocks to json file on phone.

Transfer files to PC.

Use small html/js/css file to position frames on top of original image.

badDiv.style.cssText = `top: ${value.frame.top}px; left: ${value.frame.left}px; height: ${value.frame.height}px; width: ${value.frame.width}px`;

goodDiv.style.cssText = `top: ${value.frame.left}px; right: ${value.frame.top}px; height: ${value.frame.width}px; width: ${value.frame.height}px`;

Screenshot 2024-11-05 at 08 43 01

The red bordered blocks (badDiv) are messed up.

The green bordered blocks (goodDiv) with the properties switched around are in the correct positions.

a7medev / react-native-ml-kit