The eval_kitti software contains tools to evaluate object detection results using the KITTI dataset. The code is based on the KITTI object development kit. This version is useful to validate object detection approaches before submitting results to the official KITTI ranking.
Build evaluate_object with CMake:
mkdir build
cd build
cmake ..
make
The evaluate_object
executable will be then created inside build
. The following folders are also required to be placed there in order to perform the evaluation:
data/object/label_2
, with the KITTI dataset labels.lists
, containing the .txt
files with the train/validation splits. These files are expected to contain a list of the used image indices, one per row, following these examples.results
, in which a subfolder should be created for every test, including a second-level data
folder with the resulting .txt
files to be evaluated.eval_litti/build/results
with the name of the experiment. Create a symbolic link to the folder containing your resulting txt files inside the newly created folder and rename it as data
; e.g., if the experiment is named exp1
, txt files to be evaluated should be accessible at eval_litti/build/results/exp1/data
.eval_kitti/build
and run evaluate_object
with the name of the experiment and the txt file containing the validation split as arguments. For instance, if you created the folder eval_litti/build/results/exp1
and are using the valsplit
validation set from here, you should run ./evaluate_object exp1 valsplit
.eval_kitti/parser.py
to specify the classes and parameters for which you want average results; by default, you will obtain the same stats as in the KITTI benchmark. Then, go back to eval_kitti
and run ./parser.py
passing the name of the experiment as an argument. In our example, python parser.py exp1
. Average stats (AP for detection, AOS for orientation, etc.) will be printed.This work is a derivative of The KITTI Vision Benchmark Suite by A. Geiger, P. Lenz, C. Stiller and R. Urtasun, used under CC BY-NC-SA. Consequently, code in this repository is published under the same Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License. This means that you must attribute the work in the manner specified by the authors, you may not use this work for commercial purposes and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same license.