Closed Marius-Adam closed 1 week ago
maintenance-hans[bot]
commented
2 weeks ago Guten Tag, Hans here.
[!NOTE] New features, bugfixes, updates and other improvements are all handled mostly by
@mrousavyin his free time. To support@mrousavy, please consider 💖 sponsoring him on GitHub 💖. Sponsored issues will be prioritized.
christophemenager
commented
2 weeks ago I do have the same issue, only on iOS
Marius-Adam
commented
2 weeks ago I do have the same issue, only on iOS
IOS for me too
emilje
commented
1 week ago Same, iPhone 11.
frabanca
commented
1 week ago Same issue on iPhone 14 Pro and "react-native-vision-camera": "^4.5.0".
Marius-Adam
commented
1 week ago Blocking use of any frame processor with text recognition @mrousavy
mrousavy
commented
1 week ago Blocking use of any frame processor with text recognition @mrousavy Please stop pinging me, I am not free 24h consultancy
The problem is with a Frame Processor Plugin you are using, not with my library.
I explained how Frame.orientation works in https://github.com/mrousavy/react-native-vision-camera/pull/3077, this is now finally the last change to the Frame.orientation prop and should give the plugin developer all they need to make it work.
In the case of react-native-vision-camera-text-recognition, he needs to update this switch statement with the correct values now. For example 0 is .up (as in the documentation) - not .right. With the current code it wouldn't have worked in selfie cameras or iPads.
If a Frame Processor Plugin wants to actually support rotation, they just need to add a parameter to their plugin where you can pass the current outputOrientation of the Camera.
cc @gev2002
frabanca
commented
1 week ago Hi @mrousavy , thanks for your reply and your time!
As often happens, your comments are very useful and allow us to understand the situation and in some cases manage it.
For example, your previous comment was very useful for me, because although I use another engine for OCR (this one: https://www.npmjs.com/package/@ismaelmoreiraa/vision-camera-ocr), it It was easy to find the file to edit in the plugin source (this file https://github.com/gev2002/react-native-vision-camera-text-recognition/blob/758041042d6608368fe2a41c38e3368c04f6b9c0/ios/VisionCameraTextRecognition.swift#L146-L151 a line 142) and insert the .right value instead of the .up value.
This small change - which I'm still testing - seems to be enough to solve the problem.
Thanks again for your time.
mrousavy
commented
1 week ago Thank you.
Yea the problem again was that Frame.orientation was already adjusted to output orientation before - and if you did that, it was impossible to reverse it for Skia based FPs. So we now just provide the raw value, and users can rotate it if they want to. Otherwise it only works in portrait :)
frabanca
commented
1 week ago Seems a good idea to give the plugin the raw value. Maybe - to prevent incompatibility - you can pass both so you can allow legacy plugin to work like before.
mrousavy
commented
1 week ago Well it's both the same name unfortunately.
I could rename it to something like bufferOrientation and keep the old one (orientation)? I'll think about it, but to be fair, no frame processor plugin truly supported orientation before anyways.
frabanca
commented
1 week ago Considering how easy is to "fix" the plugin(s) code, maybe it's better to leave the property name "orientation" and let:
alamothe
commented
2 days ago We are using @ismaelmoreiraa/vision-camera-ocr and it has the same problem
Flocurry
commented
2 days ago Hello @mrousavy,
I've installed react-native-vision-camera-text-recognition@3.0.4
In the file android/src/main/java/com/visioncameratextrecognition/VisionCameraTextRecognitionModule.kt, I've modified the callback function and the getFrameRotation methods
override fun callback(frame: Frame, arguments: Map<String, Any>?): HashMap<String, Any>? {
val data = WritableNativeMap()
val mediaImage: Image? = frame.image
mediaImage?.let {
val rotation = getFrameRotation(frame.orientation)
Log.d("callback FrameProcessor", "rotation: $rotation")
val image = InputImage.fromMediaImage(it, rotation)
val task: Task<Text> = recognizer.process(image)
try {
val text: Text = Tasks.await(task)
if (text.text.isEmpty()) {
return WritableNativeMap().toHashMap()
}
data.putString("resultText", text.text)
data.putArray("blocks", getBlocks(text.textBlocks))
it.close()
return data.toHashMap()
} catch (e: Exception) {
it.close()
e.printStackTrace()
return null
}
}
return null
}
private fun getFrameRotation(orientation: Orientation): Int {
Log.d("FrameProcessor", "Orientation: $orientation")
return when (orientation) {
Orientation.PORTRAIT -> 270
Orientation.LANDSCAPE_LEFT -> 90
Orientation.PORTRAIT_UPSIDE_DOWN -> 180
Orientation.LANDSCAPE_RIGHT -> 270
}
}You can see logs below (frame.orientation are same if I rotate the device)
I don't understand why my frame is not rotated in my app
You can see below my screenshots
Camera without frameProcessor props :
Camera with frameProcessor props :
Below is the frameProcessor code
import { useTextRecognition } from 'react-native-vision-camera-text-recognition';
const options = { language: 'latin' };
const { scanText } = useTextRecognition(options);
const frameProcessor = useFrameProcessor((frame) => {
'worklet';
const data = scanText(frame);
}, []);My app is locked in landscape mode
"react-native-vision-camera": "4.5.0", "react-native-vision-camera-text-recognition": "^3.0.4",
What's happening?
I want to scan text in portrait mode but the camera does not pick up anything only if the text is sideways or the phone is in landscape mode.
When logging out the frame.orientation it comes back as landscape-right when the phone is held in portrait
https://github.com/mrousavy/react-native-vision-camera/assets/63927286/a1e9f58d-27f5-47bf-848a-f579c03e6b49
Reproduceable Code
Relevant log output
Camera Device
Device
iPhone 12 Pro Max (IOS 17.5.1)
VisionCamera Version
4.4.1
Can you reproduce this issue in the VisionCamera Example app?
Yes, I can reproduce the same issue in the Example app here
Additional information