✨ Include details about the entire frame in codeScanner.onCodeScanned

[MortenRickiRasmussen]

What feature or enhancement are you suggesting?

When using the new codescanner (Which is awesome be the way) it would be great to have access to information about the entire frame. Currently the onCodeScanned function includes a Code[] which all includes their x, y, height and width details. It would be nice to have a second parameter which would include height and width details about the entire frame.

This information would allow the implementation of a scan area or selection of the most "correct" code found in a Frame if multiple codes are found in a single frame

What Platforms whould this feature/enhancement affect?

iOS, Android

Alternatives/Workarounds

Currently i have no workaround

Additional information

• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar feature requests in this repository and found none.

[mrousavy]

Which is awesome be the way

Thanks!

It would be nice to have a second parameter which would include height and width details about the entire frame.

Why exactly do you need access to that?

scan area or selection of the most "correct" code found in a Frame

I don't really understand that part..

[MortenRickiRasmussen]

The height and width information of the entire frame would make it possible to make an "scan area" on scren, and only use scanned codes found in that area

[NTKrieger]

@MortenRickiRasmussen I believe what you're talking about is a region of interest...it's implemented but not documented. For example, you can pass a third property at init.

const codeScanner = useCodeScanner({ codeTypes: ['upc-e', 'code-39'], onCodeScanned: codes => { console.log(Scanned ${codes.length} codes!) }, regionOfInterest: { x: 0, // x origin y: 0, // y origin width: 1, height: 1, }, })

I'm not sure how to format the coordinates offhand, you may have to experiment a bit.

[mrousavy]

it's implemented but not documented

@NTKrieger wdym it's not documented? Here's the docs: CodeScanner.regionOfInterest (yea it's only API docs, I didn't feel like this deserves it's own section in the docs)

Also, this only works on iOS. The native Android API doesn't support this.

[mrousavy]

The height and width information of the entire frame would make it possible to make an "scan area" on scren

@MortenRickiRasmussen what do you mean? The width and height are already provided: Code.frame

"relative to the Camera Preview" here means that this is relative to the View. If your view is 1920px in height, and the frame is at 1000px, then it is roughly at the center. Just use this as a coordinate system

[alexglazkov9]

@mrousavy regarding this:

"relative to the Camera Preview" here means that this is relative to the View. If your view is 1920px in height, and the frame is at 1000px, then it is roughly at the center. Just use this as a coordinate system

For some reason, scanned barcodes' frames return coordinates which can be out of bounds of the camera preview. Here I measure camera preview onLayout:

camera view {"height": 716.8571166992188, "width": 411.4285583496094, "x": 0, "y": 0}

Captured barcode's frame details:

frame {"height": 261, "width": 127, "x": 1105, "y": 207}

(It looks like barcode is captured in landscape mode, even though I capture it in portrait.) As you can see X coordinate (which is actually Y since it is rotated) is larger then the preview's height.

In this case, having entire's frame details would help to calculate where scanned barcode is relatively to the entire frame and then use this proportion to calculate where it is relative to the preview. Unless, I don't completely understand how "relative to the Camera Preview" works.

[mrousavy]

Ah, this is weird. iOS or Android? I think on iOS it should be accurate, but on Android it might be in sensor format. That's a bug then.

(It looks like barcode is captured in landscape mode, even though I capture it in portrait.)

Camera sensors are always in landscape. Orientation is not yet implemented in VisionCamera, see https://github.com/mrousavy/react-native-vision-camera/issues/1891

In this case, having entire's frame details would help to calculate where scanned barcode is relatively to the entire frame and then use this proportion to calculate where it is relative to the preview

I guess this should be Preview bounds always, then we don't need to pass another object every time.

For realtime drawing etc., Frame Processors are more powerful. The CodeScanner should remain simple, fast and easy to use

[alexglazkov9]

This is happens on Android, haven't tested much on iOS, but it seemed to be accurate based on my initial tests.

[suhailrifkykramphub]

So to confirm regionOfInterest only works on iOS and making it work on android does this mean we have to resort toframeProcessor instead of codeScanner prop in order to specify area of interest?

[mrousavy]

Yes, because it simply is not supported in the native MLKit Code Scanner API on Android.

[sathiyaraj45]

Is there any option to additionally set the border of the region of interest area?

[mrousavy]

What? There is no "border" or UI, this is a pure Camera component. You can build the UI yourself.

[sathiyaraj45]

when we have already implemented the camera component is 400x400 size of rectangle box with border. How can we create a custom UI for the regionofinterest area.