search

cheind / py-motmetrics

:bar_chart: Benchmark multiple object trackers (MOT) in Python
MIT License
1.37k stars 259 forks source link

Clarifying the MOTchallenge groundtruth and test .txt format #121

Closed pvti closed 4 years ago

pvti commented 4 years ago

Dear all, I'm trying to evaluate my tracking algorithm based on py-motmetric version 1.2 installed via pip with python3.6. I've read all the issues in this repos, also do some experiments with matlab-devkit. As #113 says, 

---------
Layout for ground truth data
    <GT_ROOT>/<SEQUENCE_1>/gt/gt.txt
    <GT_ROOT>/<SEQUENCE_2>/gt/gt.txt
    ...
Layout for test data
    <TEST_ROOT>/<SEQUENCE_1>.txt
    <TEST_ROOT>/<SEQUENCE_2>.txt
    ...
Sequences of ground truth and test will be matched according to the `<SEQUENCE_X>`
string

Then run

python -m motmetrics.apps.eval_motchallenge <GT_ROOT> <TEST_ROOT> where you replace <...> with actual paths.

Originally posted by @cheind in https://github.com/cheind/py-motmetrics/issues/113#issuecomment-668543190

And the python -m motmetrics.apps.eval_motchallenge --help yeilds: All file content, ground truth and test files, have to comply with the format described in Milan, Anton, et al. Mot16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016). https://motchallenge.net/

Yes, I read that paper. And as far as I got from the paper, both gt.txt and .txt must be written in the following format: frameID, trackID, bbox_top, bbox_left, bbox_width, bbox_height, confident score, class, unknown Where: confident score = 0 means inactive, =1 means active class = 1 means pedestrian WHAT IS THE 9-TH number means? I got nothing from the paper, also I ask on motchallenge forum but got no answer. In my case I prepare 2 files: /media/data3/EgoCentric_Nafosted/mot/gt/GH010383_5_462_968_1/gt/gt.txt contains: 1, 3, 794.27, 247.59, 71.245, 174.88, 1, 1, 1. 1, 6, 1648.1, 119.61, 66.504, 163.24, 1, 1, 1. 1, 8, 875.49, 399.98, 95.303, 233.93, 1, 1, 1. and /media/data3/EgoCentric_Nafosted/mot/test/GH010383_5_462_968_1.txt contains: 1, 3, 794.27, 247.59, 71.245, 174.88, 1, 1, 1. 1, 6, 1648.1, 119.61, 66.504, 163.24, 1, 1, 1. 1, 8, 875.49, 399.98, 95.303, 233.93, 1, 1, 1. Finally I run: python3 -m motmetrics.apps.eval_motchallenge /media/data3/EgoCentric_Nafosted/mot/gt/ /media/data3/EgoCentric_Nafosted/mot/test/ But got errors: 03:55:47 INFO - Found 1 groundtruths and 1 test files. 03:55:47 INFO - Available LAP solvers ['lap', 'scipy'] 03:55:47 INFO - Default LAP solver 'lap' 03:55:47 INFO - Loading files. Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/minhkv/.local/lib/python3.6/site-packages/motmetrics/apps/eval_motchallenge.py", line 121, in <module> main() File "/home/minhkv/.local/lib/python3.6/site-packages/motmetrics/apps/eval_motchallenge.py", line 101, in main gt = OrderedDict([(Path(f).parts[-3], mm.io.loadtxt(f, fmt=args.fmt, min_confidence=1)) for f in gtfiles]) File "/home/minhkv/.local/lib/python3.6/site-packages/motmetrics/apps/eval_motchallenge.py", line 101, in <listcomp> gt = OrderedDict([(Path(f).parts[-3], mm.io.loadtxt(f, fmt=args.fmt, min_confidence=1)) for f in gtfiles]) File "/home/minhkv/.local/lib/python3.6/site-packages/motmetrics/io.py", line 321, in loadtxt return func(fname, **kwargs) File "/home/minhkv/.local/lib/python3.6/site-packages/motmetrics/io.py", line 83, in load_motchallenge engine='python' File "/home/minhkv/.local/lib/python3.6/site-packages/pandas/io/parsers.py", line 676, in parser_f return _read(filepath_or_buffer, kwds) File "/home/minhkv/.local/lib/python3.6/site-packages/pandas/io/parsers.py", line 454, in _read data = parser.read(nrows) File "/home/minhkv/.local/lib/python3.6/site-packages/pandas/io/parsers.py", line 1133, in read ret = self._engine.read(nrows) File "/home/minhkv/.local/lib/python3.6/site-packages/pandas/io/parsers.py", line 2458, in read alldata = self._rows_to_cols(content) File "/home/minhkv/.local/lib/python3.6/site-packages/pandas/io/parsers.py", line 3113, in _rows_to_cols self._alert_malformed(msg, row_num + 1) File "/home/minhkv/.local/lib/python3.6/site-packages/pandas/io/parsers.py", line 2872, in _alert_malformed raise ParserError(msg) pandas.errors.ParserError: Expected 12 fields in line 1, saw 17. Error could possibly be due to quotes being ignored when a multi-char delimiter is used. Please help! @cheind @jvlmdr @smidm @hakanardo @Borda @ Any recommendation is appreciated! Thanks.

GeekAlexis commented 4 years ago

Any update on this? I ran into the same problem, I followed the format here https://motchallenge.net/instructions/

pvti commented 4 years ago

Hi, @GeekAlexis at last I managed to make it run. About the gt.txt, you can see in the paper of mot16. The 9th index is the percentage of visibility (not be occluded by others) of the object. Also you need to put in the Sequenx_x a file named seqinfo.ini that containes the field seq_length. Download the mot16 gt and take a look. image

Run evalTracking.py, select solver scipy in case the lapjav doesn't work.

pvti commented 4 years ago

Finally it works with some modifications.

pordeli commented 3 years ago

How did you created your seqmap file?

Should we have a folder? What is inside your text file? I have one sequence.

AbdurahmaanYuusuf commented 3 years ago

how can i create seqinfo.ini file?