thodan / bop_toolkit

A Python toolkit of the BOP benchmark for 6D object pose estimation.
http://bop.felk.cvut.cz
MIT License
376 stars 135 forks source link

BOP format and testing the toolkit #123

Closed Sar-thak-3 closed 1 month ago

Sar-thak-3 commented 2 months ago

Description: I'm currently grappling with an issue concerning the directory structure necessary for evaluating my pose estimation model using the BOP (Benchmark for 6D Object Pose Estimation) toolkit. It seems that the organization of my files isn't aligning properly with the toolkit's expectations, leading to errors and hindering accurate evaluations.

my config.py file

# Author: Tomas Hodan (hodantom@cmp.felk.cvut.cz)
# Center for Machine Perception, Czech Technical University in Prague

"""Configuration of the BOP Toolkit."""

import os

####### Basic ########

# Folder with the BOP datasets.
if "BOP_PATH" in os.environ:
    datasets_path = os.environ["BOP_PATH"]
else:
    datasets_path = r"/media/yashwant/storage/yashwant/ycb/ycbv_test_all"

# Folder with pose results to be evaluated.
results_path = r"/media/yashwant/storage/yashwant/predictions"

# Folder for the calculated pose errors and performance scores.
eval_path = r"/media/yashwant/storage/yashwant/result"

######## Extended ########

# Folder for outputs (e.g. visualizations).
output_path = r"/path/to/output/folder"

# For offscreen C++ rendering: Path to the build folder of bop_renderer (github.com/thodan/bop_renderer).
bop_renderer_path = r"/path/to/bop_renderer/build"

# Executable of the MeshLab server.
meshlab_server_path = r"/path/to/meshlabserver.exe"

I only wanted to evaluate the model on ycbv testing data, which is available at provided BOP website https://huggingface.co/datasets/bop-benchmark/datasets/tree/main/ycbv

And the /predictions folder contains single .csv file for predicted output by the model.

Also one more thing python scripts/eval_bop19.py --renderer_type=vispy --result_filenames=NAME_OF_CSV_WITH_RESULTS This command runs eval_bop19.py, instead there should be eval_bop19_pose.py

nv-nguyen commented 1 month ago

Please take a look at this document for more details about BOP format. You can also use this script to extract *.zip after downloading from HuggingFace get the dataset in BOP format.

For testing the toolkit, you can play with the MegaPose's predictions available in the toolkit. For example: python scripts/eval_bop19_pose.py --renderer_type=vispy --results_path ./bop_toolkit_lib/tests/data/ --eval_path ./bop_toolkit_lib/tests/data/ --result_filenames cnos-fastsammegapose_lmo-test_16ab01bd-f020-4194-9750-d42fc7f875d2.csv --num_worker 10

Feel free to re-open the issue if the error persists.