adding suport for reading .mat and .xml format annotations given by UA-DETRAC challenge.

cheind / py-motmetrics

:bar_chart: Benchmark multiple object trackers (MOT) in Python

MIT License

1.37k stars 258 forks source link

adding suport for reading .mat and .xml format annotations given by UA-DETRAC challenge. #53

Closed muaz-urwa closed 4 years ago

muaz-urwa commented 5 years ago

UA-DETRAC is an important multiple object tracking benchmark which focuses on road traffic scenarios. It is now a part of "AI City Challenge".

Challenge requires you to produce results in "MOT Challenge" format but the ground truth annotations are only available in .XML and .MAT formats which structure very different from MOT format. I had to spend several hours to parse these files and convert them into MOT format so that I can use motmetrics.

I have made following additions to support UA-DETRAC benchmark:

added new formats and relevant file loaders to io.py
updated the requirements file.
created a new app specifically for evaluating py-motmetrics for U-DETRAC challenge.

Please review and add if you think it is useful. Let me know if you need any modifications.

Regards

cheind commented 4 years ago

This sounds useful, thank you! Would you mind changing the merging destination to develop branch?

muaz-urwa commented 4 years ago

Sure, done.

cheind commented 4 years ago

Thanks! Also is there any unit test you could add to tests\ that would prove correctness of execution for future versions of motmetrics? I.e for MOT16 I added a test case based on a small sample input file (.xml, .mat in your case) with expected results.

muaz-urwa commented 4 years ago

Ah sure, I have added similar unit test for detrac data loaders.

cheind commented 4 years ago

thanks, merged. will be released with next version.

fguney commented 4 years ago

Are there any plans to integrate the aggregated tracking metric used in DETRAC overall detection thresholds, i.e. 0:0.1:1? This way, it is not comparable to official results in the benchmark.

cheind commented 4 years ago

linking this to #84

goldentimecoolk commented 2 years ago

Hi, have you finished the evaluation under the detrac-format? I'm not sure where to control the detection thresholds. I guess it should be the confidence threshold in NMS during detection postprocessing. Can you help me? Any other feedback is also welcome. Thank you.