Cannot see the outer line of the recognized image when debugging

[pdiem]

Hi, I set the debug=true but can't see the outer lines of the recognized image with the code: Imgproc.line(img_matches, new Point(scene_corners.get(0, 0)), new Point(scene_corners.get(1, 0)), new Scalar(0, 255, 0), 4); How can I see this rectangle?

[Udanita]

Did you changed in both line 86 and line 295 of java file?

[pdiem]

Yes I did. I use it with ionic 3.

[Udanita]

Java file has the method to make the lines but is not returned (or not used) from index.js . The Java file elaborate a Json matrix, maybe it could be possible to add thoose lines (or even only coordinates) to that matrix so we can handle the index.js to create rectangular with a function inside. So we can retrieve multiple objects and click on one of them for open the specific page. Are you able to handle the code for doing this?

[pdiem]

Hi Udanita, I guessed the lines can't be shown because they're drawn on the Mat which isn't added on the SurfaceView. I also tried to return these coordinates here to ionic's source code so that I can draw a 3D object with ThreeJs, but these coordinates look unrelated to ThreeJs' coordinate. Thus, now I'm trying to convert 2D-3D coordinates with OpenCV's function solvePnP (with no success yet). I'm quite new to both OpenCV and ThreeJs, so I'm not sure I'm doing the right thing; or other people can see the lines, except me. Could you give me some advises on it? Thank you so much!

[pdiem]

I'm thinking of 2 options:

1. Return the coordinates from plugin and draw 3D object in Ionic app with Threejs
2. Draw the 3D object in plugin code with OpenGL I don't know which one is more realistic and has better performance. Do you have any idea?

[Udanita]

Hi pdiem and thanks for the reply. I'm sorry but i'm not skilled so much on this. If you are able to get coordinates back to that Json array callback, i think we can add an image pointing an url directly from index.js on that coordinates so, if you have multiple objects detected, you can insert the jpg with a "click me" inside and click only to desired image. What do you think? p.s. I'm sure that @a31859 can solve in minutes.

[pdiem]

I'm try doing that, but sometimes feeling lost lol Thanks!

[Udanita]

imgDetectionPlugin.isDetecting(function(success){ console.log(success); var resp = JSON.parse(success); alert("Index rilevato: " + resp.index + ", Immagine rilevata: " + indexes[resp.index]);

I think that you should pass coordinates inside that "success" from java file and retrieve from that resp into the index.js, am i right?

[pdiem]

Yes you're right. I passed coordinates back to ionic app already. I drew a rectangle with these coordinates with Threejs, but still can't see it. From what I understand, these coordinates don't match with Threejs coordinates. I'm making a change on java file to transform the coordinates to 3D coordinates with solvePnP function, but not successful yet.

[Udanita]

you should add to canvas

[pdiem]

Will do it. Thanks a lot Udanita!

[Udanita]

Please post code if you solve. Thanks to you, pdiem!!!!

[pdiem]

Yes sure!

[a31859]

Hey @pdiem, @Udanita the rect code was used temporarily while developing the plugin but the code that showed the rect was removed due to being too slow (average of 4/5 fps).

@pdiem the issue you are having with not being able to see the coordinates is because you need to adjust the coordinates values. The coordinates are obtained from the images captured by the camera but these images have a different size than the screen view you see thus there's the need to adjust them to match the proper value in the screen view.

I've updated the plugin to return the coordinates and center/centroid of the rect in the isDetecting success callback. You'll get something like this:

{
  "message": "pattern detected", 
  "index": 0,
  "coords": {
    "1": { "x": 170.839401, "y": 181.510239 }, 
    "2": { "x": 293.745239, "y": 180.525345 }, 
    "3": { "x": 301.409363, "y": 352.518280 }, 
    "4": { "x": 171.494492, "y": 360.808655 }
  }, 
  "center": {
    "x": 234.956223, "y": 268.231842
  }
}

[pdiem]

Thanks a lot, @a31859, I'll try it out!