search

cheind / py-motmetrics

:bar_chart: Benchmark multiple object trackers (MOT) in Python
MIT License
1.37k stars 258 forks source link

Problems with understanding all the symbols #144

Open siadajpan opened 3 years ago

siadajpan commented 3 years ago

I'm having some problems with matching the symbols to the description. So far I got this:

IDF1 - ID measures: global min-cost F1 score. IDP - ID measures: global min-cost precision. IDR - ID measures: global min-cost recall. Rcl (recall) - Number of detections over number of objects. Prcs (precision) - Number of detected objects over sum of detected and false positives. GT - number of objects MT (mostly tracked) - Number of objects tracked for at least 80 percent of lifespan PT (partially tracked) - Number of objects tracked between 20 and 80 percent of lifespan ML (mostly lost) - Number of objects tracked less than 20 percent of lifespan. FP - Total number of false positives (false-alarms). FN - Total number of misses. IDs - Total number of unique object ids encountered. FM (number of fragmentations) - Total number of switches from tracked to not tracked. MOTA - Multiple object tracker accuracy. MOTP - Multiple object tracker precision. IDt - IDa IDm

Can you tell me if that is correct and what are IDt, IDa and IDm? Or describe it please in the README file

cheind commented 3 years ago

Hey,

they seem to be MOTChallenge aliases for other metrics:

https://github.com/cheind/py-motmetrics/blob/6597e8a4ed398b9f14880fa76de26bc43d230836/motmetrics/io.py#L379

'num_transfer': 'IDt',
'num_ascend': 'IDa',
'num_migrate': 'IDm',

Looking up the docstrings of keys in https://github.com/cheind/py-motmetrics/blob/6597e8a4ed398b9f14880fa76de26bc43d230836/motmetrics/metrics.py reveals

"""Total number of track transfer."""
"""Total number of track ascend."""
"""Total number of track migrate."""

PRs that fix the docs would be welcome. I believe running the script above the descriptions should yield an updated list of functions.

sieumap43 commented 1 year ago

I noticed that you have IDs as IDs - Total number of unique object ids encountered.

but the IDs is actually ID switches. https://github.com/cheind/py-motmetrics/blob/6597e8a4ed398b9f14880fa76de26bc43d230836/motmetrics/io.py#L375

Just for someone who also comes across this issue.

cheind commented 1 year ago

@sieumap43 that's a naming convention in Identification Metrics (ID) and was first used in the matlab-devkit. We sticked to it.