Adding new detection features eg lines, triangles, circles,..

[meDerek18]

I would like to see other detection shapes. I am willing to do some work to get this. However I would need some guidance. I have been teaching myself Flutter for the last year or so. I have never written a plugin or a library. Would anyone like to give me hints and guides on how to proceed. For example, years ago I changed a gear box on a Triumph spitfire. Every evening or so, someone would tell me what to do, then I would spend the next day or so doing it. The car did work for many years after that...actually it was a clutch, maybe a bit easier, but the same idea

[meDerek18]

Maybe all I need is help getting at detecting (straight-ish) edges (lines). That would be the basics for any shape. Circle could be later. By straight-ish, I mean that in an image a straight line would not be exactly straight, and presumably any line would be blurred to some extent. Currently I have started to look at the Plugin source code. If I find anything, I would have to look at how to re-generate a plugin that just detected edges

[MTschannett]

Sorry for the late response. Following is the insight I can give you at the moment. If you want I can help you more when you want to make some progress here.

So basically what you are looking for is called computer vision. This is the field, where someone tries to extract features and information out of images.

The basic pipeline is to first transform the image into a better format and then try to find shapes of all kinds in the image.

A example workflow, like in this library is to take the image

1. scale it down a bit (less pixels means less computation)
2. let a edge detection kernel run over it (kennys edge detection)
3. reduce noice by smoothing some edges (smaller edges will dissappear with this making computation faster)
4. Run a algorithm over the resulting image that extracts features from the image (hough transform)
5. Find the fitting shape

For example if you want to extract a document of a image, it would be the largest rectangle in the image, with this knowlegde and the pipline above we could find it.

A triangle could also be found with the hough transform, by looking for shapes that have three cutting edges.

There are also implementations for cirlces using hough transform (hough transform is good in finding lines in images,

In this library I tried using OpenCV to help with the computation and implementation of the hough transform.

[meDerek18]

Thanks for your response. I had left edge detection without much further that when I asked the question. ( doing something on tensor flow lite, and image classification on game cards), but I would like to get back to edge detection. I will get back to you in a couple of daysThanksDerek Davidson Sent from Mail for Windows 10 From: Marco TschannettSent: Monday, December 2, 2019 6:04 PMTo: MTschannett/image_feature_detectorCc: meDerek18; AuthorSubject: Re: [MTschannett/image_feature_detector] Adding new detection features eg lines, triangles, circles,.. (#9) Sorry for the late response. Following is the insight I can give you at the moment. If you want I can help you more when you want to make some progress here.So basically what you are looking for is called computer vision. This is the field, where someone tries to extract features and information out of images.The basic pipeline is to first transform the image into a better format and then try to find shapes of all kinds in the image.A example workflow, like in this library is to take the imagescale it down a bit (less pixels means less computation)let a edge detection kernel run over it (kennys edge detection)reduce noice by smoothing some edges (smaller edges will dissappear with this making computation faster)Run a algorithm over the resulting image that extracts features from the image (hough transform)Find the fitting shapeFor example if you want to extract a document of a image, it would be the largest rectangle in the image, with this knowlegde and the pipline above we could find it.A triangle could also be found with the hough transform, by looking for shapes that have three cutting edges.There are also implementations for cirlces using hough transform (hough transform is good in finding lines in images,In this library I tried using OpenCV to help with the computation and implementation of the hough transform.—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe. 

[MTschannett]

Nice to hear that ^^. Thanks for your response

[meDerek18]

What I did was create a Flutter Plugin project in Android studio. I did get some plugin basics to work showing that I could connect to the plugin and return data. What I found was that I had to open the project in 2 different ways, depending on what I was changing: 1. as a Android Studio Flutter project that got to the Dart code in the example subdirectory, and 2. as an Android Studio Java project that could get at the Java code under the project level android directory.

I copied your code into my project(s).

From my notes when working on this I was getting an error message: no implementation found for long org.openvc.core.Mat.n_mat(). I assumed that I was not getting at openCV correctly.

Using information from Elvis Chidera (Aug 2018) https://android.jlelse.eu/a-beginners-guide-to-setting-up-opencv-android-library-on-android-studio-19794e220f3c I placed the openVC module in the Android Java project. I had difficultieswith openVC v 4.0.1 and v3.3.1 allowed me to go futher. I noticed that your example did not appear to have and openCVLibrary in the android directory. Maybe this is not the best way to get at openVC.

I am getting a message: ERROR: Cannot read packageName from G:\AndroidStudioFlutter\my_test_plugin\android\openCVLibrary331\AndroidManifest.xml Affected Modules: openCVLibrary331

I will start again by re-creating a plugin, and adding you code.

[meDerek18]

Found that do not have to open "Java" code part of project as another Project. Can find JAVA code files by File -> Open -> going to my_plugin2\android\src\main....MyPlugin2Plugin.java

[meDerek18]

I Downloaded the Github project image_feature_detector-master. Run attempt in Android Studio gives messages below. Maybe it is confued between the Java part of this project having a package name com.matas.image_feature_detector and there being a real plugin with the same name. I will continue with my plugin project created with Android Studio, and start adding code to use openCV

E/MethodChannel#image_feature_detector( 4524): Failed to handle method call E/MethodChannel#image_feature_detector( 4524): java.lang.IllegalStateException: Reply already submitted E/MethodChannel#image_feature_detector( 4524): at io.flutter.embedding.engine.dart.DartMessenger$Reply.reply(DartMessenger.java:124) E/MethodChannel#image_feature_detector( 4524): at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler$1.notImplemented(MethodChannel.java:214) E/MethodChannel#image_feature_detector( 4524): at com.matas.image_feature_detector.ImageFeatureDetectorPlugin.onMethodCall(ImageFeatureDetectorPlugin.java:40) E/MethodChannel#image_feature_detector( 4524): at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:201)

[MTschannett]

When I remember the things correct. I included opencv via a library from maven. That helped me getting rid of the whole configuration of opencv with copying the libaries and files around manually.

Basically it's this:

https://github.com/MTschannett/image_feature_detector/blob/master/android/build.gradle#L37

E/MethodChannel#image_feature_detector( 4524): Failed to handle method call
E/MethodChannel#image_feature_detector( 4524): java.lang.IllegalStateException: Reply already submitted

I think i got this error in a few cases, when repeatedly calling the plugin (i think 5-6 times then this error happended once) after that i had to restart the test application to get it running again.

I don't know if copying around the files is the most efficient way ^^.

I would try to clone this repository, all the dependencies should either be loaded via gradle or pub when working on the "dart side" of things.

To open the project you can (assuming you are using vs code) right click and click open in android studio in the context menu.

[meDerek18]

This source says that the version of openCV should match the Android Studio version. https://www.youtube.com/watch?v=pzuwrYgOnDQ . I cannot find any other mention of this, but I did find that opencv3.3.1 did get further than when I tries opencv 4.0.1 (my Android Studio is 3.4.1).

[meDerek18]

I downloaded the project from Github. The screen shows: Running on: 4.0.1 so I assume it is connecting to openCV, but "No contour calculated". The messages I get are: I/OpenGLRenderer(19078): Initialized EGL, version 1.4 E/MethodChannel#image_feature_detector(19078): Failed to handle method call E/MethodChannel#image_feature_detector(19078): java.lang.IllegalStateException: Reply already submitted E/MethodChannel#image_feature_detector(19078): at io.flutter.embedding.engine.dart.DartMessenger$Reply.reply(DartMessenger.java:124) E/MethodChannel#image_feature_detector(19078): at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler$1.notImplemented(MethodChannel.java:214) E/MethodChannel#image_feature_detector(19078): at com.matas.image_feature_detector.ImageFeatureDetectorPlugin.onMethodCall(ImageFeatureDetectorPlugin.java:40) E/MethodChannel#image_feature_detector(19078): at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:201) E/MethodChannel#image_feature_detector(19078): at io.flutter.embedding.engine.dart.DartMessenger.handleMessageFromDart(DartMessenger.java:88) E/MethodChannel#image_feature_detector(19078): at io.flutter.embedding.engine.FlutterJNI.handlePlatformMessage(FlutterJNI.java:219) E/MethodChannel#image_feature_detector(19078): at android.os.MessageQueue.nativePollOnce(Native Method) E/MethodChannel#image_feature_detector(19078): at android.os.MessageQueue.next(MessageQueue.java:323) E/MethodChannel#image_feature_detector(19078): at android.os.Looper.loop(Looper.java:135) E/MethodChannel#image_feature_detector(19078): at android.app.ActivityThread.main(ActivityThread.java:5417) E/MethodChannel#image_feature_detector(19078): at java.lang.reflect.Method.invoke(Native Method) E/MethodChannel#image_feature_detector(19078): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:764) E/MethodChannel#image_feature_detector(19078): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:626) E/DartMessenger(19078): Uncaught exception in binary message listener E/DartMessenger(19078): java.lang.IllegalStateException: Reply already submitted E/DartMessenger(19078): at io.flutter.embedding.engine.dart.DartMessenger$Reply.reply(DartMessenger.java:124) E/DartMessenger(19078): at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:219) E/DartMessenger(19078): at io.flutter.embedding.engine.dart.DartMessenger.handleMessageFromDart(DartMessenger.java:88) E/DartMessenger(19078): at io.flutter.embedding.engine.FlutterJNI.handlePlatformMessage(FlutterJNI.java:219) E/DartMessenger(19078): at android.os.MessageQueue.nativePollOnce(Native Method) E/DartMessenger(19078): at android.os.MessageQueue.next(MessageQueue.java:323) E/DartMessenger(19078): at android.os.Looper.loop(Looper.java:135) E/DartMessenger(19078): at android.app.ActivityThread.main(ActivityThread.java:5417) E/DartMessenger(19078): at java.lang.reflect.Method.invoke(Native Method) E/DartMessenger(19078): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:764) E/DartMessenger(19078): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:626)

[meDerek18]

wow!!! something worked. I think it might have been my own changes that caused the problem. I will have a look to see if I can see what made it give results

[meDerek18]

I still have a question on how I should edit the Java project in the android\src\main... directory. I had found that I could open this project as an Android Studio "java" project, make changes, and (apparently) it all got compiled when I opened and ran the Flutter plugin project. When things were not working, I opened the JAVA source files in the Flutter plugin project with File->Open and drilled down the directories to the JAVA file. The problem here is that it gives lots of error messages in the edit file. I presume this is because Android Studio thinks it is a flutter project, and all these Java-type libraries are not known.

[meDerek18]

I am now editing the Java code by opening an Android Studio project on the Android subdirectory. It gives messages like cannot work out what io.flutter.plugin.... and MethodCallHandler mean, but it does not seem to matter. The whole plugin compiles when opened as a Flutter project. Now comes the interesting bit: working out what all these Imgproc type calls mean

[MTschannett]

Sorry for the late answer.

Yes you are on the right track. There are two platforms involved here. IOS writen in Objective C and the android part written in Java.

To edit the Java code you have to open it in Android Studio. After that you need to compile the whole app, because I believe Flutter doesn't reload these changes.

The Imgproc calls are calls to OpenCV, which are named like these. The best source for of documentation is the OpenCV page. They are all called the same as their C++ counterpart so you could also use documentation written for e.g. Python or C++. They only differ in slight ways.

Your error seems like the dependencies weren't synced. I believe this can be done in Android Studio (something with Gradle). But my Gradle knowledge is not the best :/