search

LocalJoost / BlogComments

This repo is solely used for comments on https://localjoost.github.io/
4 stars 0 forks source link

Comments on "HoloLens AI - training a YoloV8 model locally on custom pictures to recognize objects in 3D space" #452

Open LocalJoost opened 1 year ago

LocalJoost commented 1 year ago

Original article: https://localjoost.github.io/HoloLens-AI-training-a-YoloV8-model-locally-on-custom-pictures-to-recognize-objects-in-3D-space/

aforadil commented 1 year ago

Is it possible to deploy this project in Quest 2? Actually I am doing some project for object detection in VR with unity. Does it work on 2D images/video Placed in a 3D space Scene?

LocalJoost commented 1 year ago

Well it's recognizing objects based upon 2D pictures taken from reality. I am not sure if you can make 2D snapshots of the 3D view in a Quest, but if you could, I guess it could work. But I would kind of fail to see the point.

gelsogrove commented 6 months ago

Hello amazing experiment, i am wondering how did you get the profondity because the computer vision return only x and y but doesn't return the z thanks

LocalJoost commented 6 months ago

Thank you for the compliments @gelsogrove. You are correct. I only get X and Y. There is a link in the article linking back to my very first article about computer vison in 2018 that explains the trick. Summary - pretend that picture hangs in front of you, calculate where the Yolo box then is in space, shoot a ray through the center of the Yolo box and where it hits the spatial map, that's probably where the object is.

gelsogrove commented 6 months ago

amazing tips...the last question i am having some isse to load the onnx model inside the code (unity), and can't find a good plugin or a good code example :-( do you have any tips?

LocalJoost commented 6 months ago

@gelsogrove first line of my post: In April, I described my experiments with computer vision, using a HoloLens 2, the Unity Barracuda inference engine,

So use Barracuda :)

Dia-su commented 4 months ago

Hej, Thank you for this project! I was looking for something exactly like this. Unfortunately, I cannot get it to work on the HoloLens. The debug option with your debugger object DebugDisplayer in Unity works fine and does detect airplanes on a webcam. I can build the project (as you mentioned: only with DebugDisplayer deactivated) and deploy it to the HoloLens without errors, but then in the HoloLens the app does not start. I also tried Holographic Remoting (from the MRTK) inside Unity, but nothing seems to reach the HoloLens. Do you have some ideas what the problem could be? Any sugestions might help :)

LocalJoost commented 4 months ago

Hi @Dia-su thanks for the compliments

First of all, it does not work with Holographic remoting. It still uses your webcam. As to deployment issues, it can be a lot of things.

It works, it really works. Multiple people have already gotten it to work :)