Closed sfchen94 closed 1 year ago
Unfortunately different papers/datasets came up with slightly different implementations of the same metric. Thus, there are some small differences such as including or excluding invalid frames depending on the benchmark. In the end, we made sure that the results produced by our implementation can reproduce the results reported in the individual papers (LaSOT, OTB, NFS, UAV etc). Maybe we should not expose this to the user and simply select these parameters automatically depending on the datasets that is used for evaluation...
Hi, Thanks for your work, I am curious why some datasets handle invalid frames when reporting the evaluation results, but others do not. For example, there exist different evaluation results when we handle the Invalid frames on LaSOT. But most of your work skips to handle invalid frames on this dataset. Is there any reference to explain that?