how to evaluate multi-class multi-object 2D-tracking task on custom dataset

JonathonLuiten / TrackEval

HOTA (and other) evaluation metrics for Multi-Object Tracking (MOT).

MIT License

963 stars 239 forks source link

how to evaluate multi-class multi-object 2D-tracking task on custom dataset #64

Open upsx opened 3 years ago

upsx commented 3 years ago

i try to use run_mot_challenge.py tool to evaluate my tracker on my custom dataset. Before evalutation, i have change default_dataset_config['CLASS_TO_EVAL'], self.valid_classes, self.class_name_to_class_id, distractor_class_name etc in mot_challenge_2d_box.py. But the output MOTA is negative, and i dont know how to solve it. Can you help me？thks

Jason-lzc commented 2 years ago

顶，同求

orilifs commented 2 years ago

Also interested please.

jvlmdr commented 2 years ago

Negative MOTA does not necessarily indicate a problem with the evaluation. MOTA is negative when FN + FP > GT, i.e. when FP > GT - FN = TP (described in the HOTA paper https://arxiv.org/abs/2009.07736; "MOTA scores can be negative and are unbounded."). Check whether CLR_TP, CLR_FN, CLR_FP seem reasonable.