Is it possible to run on customized data?

[cwchenwang]

Thanks for the great work! I am wondering if the code can be used for in the wild data with CAD models and images to test?

[cwchenwang]

According to the README, I rendered templates with zs6d_template_rendering repo. However, I don't know how to prepare your_template_config.json, could you provide more information on that?

[PhilippAuss]

Thanks for the interest in our work! When you have the templates you have to set up the cft_template_gt_generation_yourdata.json. There you have to specify the path to the template corresponding ground truth poses obj_poses.npy, as well as the output paths for the descriptors. Additionally, you need to create a models_info.json file. These files are part of BOP datasets: https://bop.felk.cvut.cz/datasets/ There you can check the format of such a file. It contains an id for each object model with the diameter, min_x and annotated symmetries. You have to create this for your objects. If you want to use ZS6D you need a detection pre-stage, in my experiments, I use detections from CNOS: https://github.com/nv-nguyen/cnos This is a zero-shot object segmentation and classification method based on SAM and templates, so you can use the same templates for both. Alternatively, you can train a segmentation method like Yolo or Mask_RCNN: https://github.com/matterport/Mask_RCNN But for these two you have to train with annotated or synthetic training images. I hope this answers your question!