Opencv with AR.js

[y0un35]

Excuse me, i didn't know where to ask, I developped an algorithm to detect fingers using Opencv iwth Python, and i want to know if it's possible to combine my python script with AR.js, the goal is to get some coordinates from my python and send it to AR.js in order to put a ring in those coordinates!. Thanks in advance

[kalwalt]

HI @y0un35 I know the Brython project but i don't think that your OpenCV python code , can be injected directly in Brython... because for sure you can't import specific python libs as OpenCV or for example numpy, but it seems you can instead import javscript libs https://brython.info/static_doc/en/jsobjects.html. Interesting task, I hope that you can do this! Good luck :smile:

[kalwalt]

You can also import python modules read here: https://brython.info/static_doc/en/import.html

[Ryuno-Ki]

Hm, I wonder, whether Tensorflow.js could be used here. CV stands for computer vision, doesn't it? (Curriculum Vitae is another common meaning …).

They have some demos online.

[evaristoc]

@y0un35

I developped an algorithm to detect fingers using Opencv iwth Python,

Looks like you are trying an algo that would read your finger position and that should reflect into a position in a canvas? Virtual control?

the goal is to get some coordinates from my python and send it to AR.js in order to put a ring in those coordinates!.

If I am right about the virtual control stuff, you might not need AR.js per se. AR.js is for marker-based AR. I might get it wrong but I don't see any requirement for markers in your project?

Basically, a simple html canvas will do as long as you make the right translations. If your code works as you expect, you can even move the object you have rendered in the destination canvas.

If you keep the source section (openCV/Python) separated from the destination one (html/javascript), you could always send the coordinates through for the destination side (a client, usually considered frontend) to consume. Possibly a json file (two servers: one for Python/openCV and one for html/AR.js) or any other format. There could be a delay. You might have to think about the appropriate server/client setup.

The solution above is general and ALSO applies to rendering a marker-based object with AR.js: you can find ways to virtually manipulate an AR object rendered with AR.js by making it to respond to remote data.

i want to know if it's possible to combine my python script with AR.js

No idea regarding this. I know there are libraries that translate from one language to another (I have used some drivers) but there is always the incurred cost of language conversion. In fact, because you are using Python instead of plain openCV you probably have already a bit of a slower performance, I guess.

Hope that helps.

[y0un35]

Thank you everyone for your reponses, i just have one question! can i consider this project as AR project?!! because the goal is to take a picture of your hand, detect phalanx of fingers then put the ring on a choosen finger, can we consider Augemented reality technology if it's applied on picture or are we taling about AR just if it's applied in video real time?!

@evaristoc : thanks for your response, yeah that's what i thought, i thing i won't need AR.js for this! especially that the user will take the picture of his hand then the picture will be sent to the server where it will detect fingers and coordinates where the ring should be put. after that i think maybe just some sort of JS code (maybe using three.js) to put the ring on those coordinates!.

@Ryuno-Ki : Yeah CV for computer vision, i'm not applying for jobs here, Lol, just kidding; well, i already finished my script of opencv, i can detect fingers and phalanx of fingers where i should put the ring, all i need now, is the front-end step! i thought maybe i can use AR.js but it seems maybe i don't need that

@Ouyang-Zhaoxing : well i already finished my opencv python script, my idea is this script will be treated like backend in the server, and for the front end i'll use AR.js! (or maybe any javascript framework)!!not specifically something of AR?!

@kalwalt : well, my idea was to make my opencv-python script works on the backend on the server, and for the front end where the ring diplay, will be with Javascript! except i do'nt know what to use anymore!! AR.js or just any simple javascript framwrok!.

[evaristoc]

@nicolocarpignoli I don't know but I think this one can be closed?

[nicolocarpignoli]

yes, thanks