search

chensong1995 / HybridPose

HybridPose: 6D Object Pose Estimation under Hybrid Representation (CVPR 2020)
MIT License
412 stars 64 forks source link

Using the QR codes using training, do they need to be present during inference? #9

Closed ishangupta3 closed 4 years ago

ishangupta3 commented 4 years ago

The aruco markers (QR codes )help provide the Rt matrix. If they are used in the training data, do they need to be also present during inference?

What would be the best way to acquire the Rt (ground truth pose matrix) with/without the qr codes in your personal opinion ?

chensong1995 commented 4 years ago

Hello ishangupta3,

Thanks for your questions!

The pipeline of HybridPose does not assume the existence of aruco markers in the background. However, it is indeed the case that both the datasets we experimented on contain those QR codes.

When the regression network is trained on images with a certain type of background, and tested on images with another type of background, there will be a generalization issue. To compensate this, you may try using some synthesized training examples, like what the authors in PVNet did.

I am not an expert in dataset labeling. However, if I were to create a brand-new 6DoF object pose dataset, I would first look into planting some small (ideally invisible) electronic markers into the object. These markers should be able to somehow use radio waves to locate their 3D positions. The 3D positions will allow me to acquire the ground-truth object pose.

I hope this helps.

ishangupta3 commented 4 years ago

Thank you @chensong1995 .