search

derenlei / Unity_Detection2AR

Localize 2D image object detection in 3D Scene with Yolo in Unity Barracuda and ARFoundation.
https://derenlei.medium.com/object-detection-with-localization-using-unity-barracuda-and-arfoundation-794b4eff02f8
MIT License
222 stars 61 forks source link

Rotation estimation #24

Closed Kanyade closed 3 years ago

Kanyade commented 3 years ago

Is there a way to estimate a detected object's rotation(custom dataset)? I know that it is not a 3D object detection project. For example with a detected object's key features, like a human's eyes but with an object's typical part, so something that is clearly associated with the front. I don't have 3D CAD Models.

derenlei commented 3 years ago

@Kanyade, that sounds like an interesting idea! I can imagine it's doable if you have 3D models to match the feature points, but it's also doable in a computer vision way if only targeting a few objects.

You can set up a dataset that includes rotation labels to train your Yolo model. Like this example label set: [person-front, person-back, car-front, car-back]

I think it will work if you have enough data.

Kanyade commented 3 years ago

I don't have 3D models, it is a great idea, thank you!