Closed ErnestoMM94 closed 3 years ago
Thanks for the question @ernestoMM94.
Evaluation requires some kind of defined evaluation methodology. For example, SLAM could be cumulative error in final pose, IOU error in final object map, or some hybrid combination.
Evaluation methods are defined using BenchBot add-ons. An add-on can define novel functionality for the system, including evaluation methods. To create your own you'll need to:
The documentation for the benchbot_addons repository should help you process, and understand how you can add custom evaluation functionality if you need.
I was wondering if evaluation performance only works with object detection? Is it possible to just focus on SLAM without object detection to evaluate the results?
Example: benchbot_batch --robot carter --task semantic_slam:active:ground_truth --envs miniroom:1 --native python my_solution.py