HengLan / LaSOT_Evaluation_Toolkit

[CVPR 2019 & IJCV 2021] LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking
http://vision.cs.stonybrook.edu/~lasot/
Apache License 2.0
110 stars 13 forks source link

关于视频帧数和所提供的结果文件不对应问题 #18

Closed Giveupfree closed 2 years ago

Giveupfree commented 2 years ago

经过对比发现paddle1序列共2703个视频序列和注释,但提供的各个跟踪器结果中却只有2500个结果。paddle10共2011个,但是提供的跟踪器结果中却有2421个结果,显然数目是不对的,且对于paddle10,多出的410个视频序列和注释官方并未提供,因此不能用来做测试。

HengLan commented 2 years ago

Hi,

Thanks for the question. In fact, for those sequences in which the length of GT does not match that of the tracking result, we automatically cut down the extra frames in the sequence.

In specific, in the file "calc_seq_err_robust.m", line 9-11 as follows:

if size(results, 1) ~= size(rect_anno, 1) results = results(1:size(rect_anno, 1), :); end

We only perform evaluations on frames that have GT.

In a word, you can simply run your tracker on the whole sequence. When evaluating the algorithm, the extra frames will be ignored (for all trackers).

Hope it helps to answer your question.

laisimiao commented 2 years ago

@HengLan But why in paddle-1/groundtruth.txt, there are 2703 anno boxes, but in toolkitv2/anno/paddle-1.txt there are 2500 anno boxes, it may introduce inconvenience into community, especially for those who use pysot-toolkit and they generate .json using annos in dataset not tooklit.

HengLan commented 2 years ago

@HengLan But why in paddle-1/groundtruth.txt, there are 2703 anno boxes, but in toolkitv2/anno/paddle-1.txt there are 2500 anno boxes, it may introduce inconvenience into community, especially for those who use pysot-toolkit and they generate .json using annos in dataset not tooklit.

Please use the anno in the evaluation toolkit for evaluation. If you want to use another third party for the evaluation of trackers, you should adapt the third party to the toolkit. I may change the paddle-1 to make it consistent in the future (thanks for pointing it out ).

Thanks.