fbernaly
commented
9 months ago We would like to keep the API of the plugin the same as its native counterpart. Any convenience method related to handling coordinates in Flutter/iOS/Android should be handle by the client app as shown in the example app as you have already noticed. If you are interested you can contribute creating a Wiki page that we could include in our repo explaining that.
Hi everyone 👋
I checked all the issues here and quite a few of them were related to single block/line etc. tap and extracting text from it. I know the response was that it is out of the scope of the library, probably it confuses people as just showing the colored borders may be good for demo but sooner or later text extraction is needed for the feature to be actually useful.
Before doing anything first I checked the method from _onTapDown from https://github.com/flutter-ml/google_ml_kit_flutter/issues/511. I noticed though that it doesn't work very well, the coordinates don't match. Then I checked https://github.com/flutter-ml/google_ml_kit_flutter/issues/507 and it became clear that those coordinates coming from boundingBox are not in fact related to screen but to image processed, that's why checking if coords from TapDownDetails (screen cords) are within boundingBox (image) won't give good results.
So the solution is to obviously convert boundingBox coords to screen cords using translateX etc. but it's all already done in text_detector_painter.dart. To do it quick and dirty I created some getter in this class returning map (block to cornerPoints), then in text_detector_view under CustomPaint, added GestureDetector, created rect from point 0 and 2, and then finally I can do a proper check if the tap coords are within tapped text rect.
My question is whether it is possible to add some extra field on TextBlock class level related to cornerPoints, although not the ones containing image coords but those from text_detector_painter related to actual screen coords? Or do anything to facilitate extracting those points, as from what I see in painter class there are some additional checks related to iOS/Android platforms that influence calculations as well. Would be very handy to get them from there without making any changes locally to the library :)