search

kolaszko / hl2_detection

Apache License 2.0
9 stars 2 forks source link

Project the bounding box from 2D to 3D #1

Open NTUZZH opened 1 year ago

NTUZZH commented 1 year ago

Hi there,

It's my honor to read your latest paper about deploying the YOLO_v8 object detection model with HoloLens2.

I have one query about back-projecting the bounding box from 2d to 3d. Assuming that we already get the output from YOLO model, which is a 2d image with detected objects and bounding box. Then we want to project this bounding box into the precise location of the 3D world in the HoloLens2's view. To realize this, we need to find the 3D coordinate of this bounding box to place it. My question is how can we get a 3D coordinate from just a 2D image from the model output? Shall we use the depth information provided by HoloLens2?

I will be so appreciate if kindly you could discuss this problem with me! ♥

Regards, -Ziheng

kolaszko commented 1 year ago

Hello,

We are using a RaycastLaser https://github.com/kolaszko/hl2_detection/blob/alpha/MachineLearningPlayground/Assets/Scripts/RaycastLaser.cs from https://github.com/EnoxSoftware/HoloLensCameraStream to find a collision of a ray with a spatial map and use the collision point as a position of a 2D bounding box.

Cheers, Mikołaj

NTUZZH commented 1 year ago

Thanks for your reply!