search

LocalJoost / BlogComments

This repo is solely used for comments on https://localjoost.github.io/
4 stars 0 forks source link

Comments on "HoloLens AI - training a YoloV8 model locally on custom pictures to recognize objects in 3D space" #452

Open LocalJoost opened 10 months ago

LocalJoost commented 10 months ago

Original article: https://localjoost.github.io/HoloLens-AI-training-a-YoloV8-model-locally-on-custom-pictures-to-recognize-objects-in-3D-space/

aforadil commented 10 months ago

Is it possible to deploy this project in Quest 2? Actually I am doing some project for object detection in VR with unity. Does it work on 2D images/video Placed in a 3D space Scene?

LocalJoost commented 10 months ago

Well it's recognizing objects based upon 2D pictures taken from reality. I am not sure if you can make 2D snapshots of the 3D view in a Quest, but if you could, I guess it could work. But I would kind of fail to see the point.

gelsogrove commented 1 month ago

Hello amazing experiment, i am wondering how did you get the profondity because the computer vision return only x and y but doesn't return the z thanks

LocalJoost commented 1 month ago

Thank you for the compliments @gelsogrove. You are correct. I only get X and Y. There is a link in the article linking back to my very first article about computer vison in 2018 that explains the trick. Summary - pretend that picture hangs in front of you, calculate where the Yolo box then is in space, shoot a ray through the center of the Yolo box and where it hits the spatial map, that's probably where the object is.

gelsogrove commented 1 month ago

amazing tips...the last question i am having some isse to load the onnx model inside the code (unity), and can't find a good plugin or a good code example :-( do you have any tips?

LocalJoost commented 1 month ago

@gelsogrove first line of my post: In April, I described my experiments with computer vision, using a HoloLens 2, the Unity Barracuda inference engine,

So use Barracuda :)