🔥🔥🔥Check out the new Oryon version technical report!
This is the repository that contains source code for the Oryon website and its implementation. This work featured as highlight paper at CVPR'24.
First of all, download oryon_data.zip
and pretrained_models.zip
from the release of this repository.
The first contains the ground-truth information and the specification of the image pairs used, the second contains the third-party checkpoint used in Oryon (i.e, the tokenizer and PointDSC).
Run setup.sh
to install the environment and download the external checkpoints.
By default all experiments folder are created in exp_data/
.
This can be modified in the config file.
Training with default settings:
python run_train.py exp_name=baseline
Run the following to obtain results with the basic 4 configurations. By default, the last checkpoint is used.
python run_test.py -cp exp_data/baseline/ dataset.test.name=nocs test.mask=predicted
python run_test.py -cp exp_data/baseline/ dataset.test.name=nocs test.mask=oracle
python run_test.py -cp exp_data/baseline/ dataset.test.name=toyl test.mask=predicted
python run_test.py -cp exp_data/baseline/ dataset.test.name=toyl test.mask=oracle
Our data is based on three publicly available datasets:
We sample scenes from each dataset to build the training and testing partition (20000 image pairs for SN6D and 2000 for REAL275 and TOYL), and report in the following folder the scene ids and image ids used for each partition.
From the repository download the test ground-truth, the object models and the data of the real_test
partition. This should result in three files: obj_models.zip
, gts.zip
and real_test.zip
Run the prepare_nocs.sh
script to unzip and run the preprocessing.
By default this will create the nocs
folder in data
, and can be changed by modifying the above script.
Download the object models and the test partition from the official BOP website:
wget https://bop.felk.cvut.cz/media/data/bop_datasets/tyol_models.zip
wget https://bop.felk.cvut.cz/media/data/bop_datasets/tyol_test_bop19.zip
Run the prepare_toyl.sh
script to unzip and run the preprocessing.
By default this will create the toyl
folder in data
, and can be changed by modifying the above script.
Download the images from the official repository of ShapeNet6D, and the object models of ShapeNet from HuggingFace.
Run the prepare_sn6d.sh
script to unzip and run the preprocessing.
Note that each image of ShapeNet6D shows a different random background, so that we consider each image as being part of a different scene. ShapeNet6D provides a map from their object ids to the object ids of the original ShapeNetSem: we use this map to associated the object name and synonym sets of ShapeNetSem to each object model in ShapeNet6D.
NB: ShapeNet6D is not currently supported for evaluation (i.e., the symmetry annotations needed by the BOP toolkit are missing).
This work was supported by the European Union’s Horizon Europe research and innovation programme under grant agreement No 101058589 (AI-PRISM), and made use of time on Tier 2 HPC facility JADE2, funded by EPSRC (EP/T022205/1).
We thank the authors of the following repositories for open-sourcing the code, on which we relied for this project:
@inproceedings{corsetti2024oryon,
title= {Open-vocabulary object 6D pose estimation},
author = {Corsetti, Jaime and Boscaini, Davide and Oh, Changjae and Cavallaro, Andrea and Poiesi, Fabio},
journal = {IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR)},
year = {2024}
}
The website template is from Nerfies.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.