search

chensong1995 / HybridPose

HybridPose: 6D Object Pose Estimation under Hybrid Representation (CVPR 2020)
MIT License
412 stars 64 forks source link

Missing CAD models and custom annotations #62

Closed lucasbrynte closed 3 years ago

lucasbrynte commented 3 years ago

Hi!

Thanks for the very interesting work!

I am trying to train HybridPose on LINEMOD according to the instructions, but receive an error for certain objects when training. It appears that some objects are missing a mesh.ply file (e.g. ./data/linemod/original_dataset/eggbox/mesh.ply).

I believe it is expected that the original LINEMOD data doesn't come with CAD models for all objects, but such models clearly seem to be expected. Could I be missing some pre-processing step? I've tried to follow the descriptions.

Thanks in advance! Lucas

lucasbrynte commented 3 years ago

I just thought I'd clarify a bit what data is and what isn't available: There is indeed a mesh file named OLDmesh.ply, and also a point cloud file named object.xyz for e.g. the eggbox model (but HybridPose seems to expect a mesh.ply file).

For the eggbox model in particular, there is a .obj file for the mesh in ./data/occlusion_linemod/models/eggbox/008.obj, but in general the occlusion linemod models don't share the same coordinate frame as the original mesh.ply files.

Furthermore, I know that BOP is providing high quality .ply files for all models: https://bop.felk.cvut.cz/datasets/

For any of the above workarounds, I am however a bit hesitant as to just using an alternative model without making sure that the object coordinate system is consistent between mesh and pose annotations.

Hoping you could shed some light :)

Cheers, Lucas

chensong1995 commented 3 years ago

Hello Lucas,

Thanks for your interest in our work! I believe the original linemod dataset is missing mesh.ply on cam and eggbox (correct me if I am wrong). For these two objects, we simply convert the object.xyz file into the .ply format.

Here is an example for cam:

Screenshot from 2021-05-24 12-58-52

On the left, we have object.xyz. On the right, we have mesh.ply. As you can see, the point coordinates match each other. This mesh.ply does not contain any edge information, neither does HybridPose use any edge information.

It is true that the occlusion linemod labels are given in a different coordinate frame than linemod. Please refer to this line for the conversion between the two coordinate frames.

I hope this helps!

lucasbrynte commented 3 years ago

Thank you Chen for the prompt reply, it's been really useful!

//Lucas