search

capawesome-team / capacitor-mlkit

⚡️ ML Kit plugins for Capacitor. Supports Android and iOS.
https://capawesome.io/plugins/mlkit/
Apache License 2.0
144 stars 42 forks source link

feat(barcode-scanning): add `cornerPoints` option #130

Open robingenz opened 8 months ago

robingenz commented 8 months ago

Plugin(s)

Current problem

Currently, all detected barcodes are passed to the webview, even if they are outside a scan area. This leads to unnecessary data being transferred between the native layer and the web layer. In addition, some logic needs to be implemented at the web layer to filter out the barcodes that are outside the scan area.

Preferred solution

Add a cornerPoints option that allows you to define a valid area on the native layer. In the first step, this should have no effect on the size of the camera view. This information should only be used to filter the barcodes on the native layer.

In the next step, something like #79 could be implemented.

Alternative options

No response

Additional context

See https://github.com/capawesome-team/capacitor-mlkit/issues/96#issuecomment-1948857243

Before submitting

robingenz commented 7 months ago

@Melynt3 Is that related to this issue?

Melynt3 commented 7 months ago

@Melynt3 Is that related to this issue?

sorry, got opened few tabs, already posted it correctly

mansours commented 3 months ago

We are doing native Android development for one of our apps, for years we were using zxing, we swtiched to ML Kit for a minute before we had to switch back because customers complained with multi-scans. ML Kit was a disaster for them, accuracy is of utmost importance for these users in their business process. The wrong scan multiple scan issue is not reproducible in the same 1:1 every time because the nature of analyzing noise in pixel information fed to each device differently from the natural world. We definitely experienced a similar result of what was shown in https://github.com/capawesome-team/capacitor-mlkit/issues/96#issue-1988968014

One of the ideas floated was to have an "accuracy mode" where a single barcode is expected, must be centred on the camera (see zxing's scan red line). the same barcode must be read over many frames and outnumbers the number of wrong values read significantly stat wise. Instead we found a zxing Android 14 compatible embedded solution and gave up on ML Kit because out of the box was more difficult and effort. That was for a native app.

We tried this version for Capacitor 6 and it gave us the same inaccurate experience for our multi platform app. We reverted to staying on Capacitor 5 for now. Not sure if you needed more community feedback or if this is the place to do it. We're in a tough spot if we rely on ML Kit to be the solution (as its currently implemented, might even be a ML Kit project ticket to say this needs an easy accuracy mode out of the box as a standard less coding config to pick from,

robingenz commented 3 months ago

@mansours Your comment is not related to this issue. Feel free to create a new issue.