search

got-10k / toolkit

Official Python toolkit for generic object tracking benchmark GOT-10k and beyond
http://got-10k.aitestunion.com/
MIT License
555 stars 95 forks source link

error 404 on evaluation server #2

Closed happyjin closed 5 years ago

happyjin commented 5 years ago

Hi: Thanks for your awesome project. The server is not available right now. Can you please fix the server? And I want to compare my results with some other trackers. Could you please tell me how to plot curves for multiple trackers in the same panel like what you have done in the paper? Thanks in advance!

happyjin commented 5 years ago

Hi: If I want to compare my model with other trackers on VOT or OTB dataset. Do you offer data for other trackers on OTB and VOT dataset?

huanglianghua commented 5 years ago

Hi, @happyjin, thanks for your interests. The links have been fixed in the new commit of this repo (master branch).

We provide GOT-10k tracking results/reports on the download page of GOT-10k website.

Currently we have no plan to gather or release OTB and VOT results. For VOT datasets, you can directly download results from its homepage, e.g., http://data.votchallenge.net/vot2018/vot2018_results.zip.

happyjin commented 5 years ago

Hi @huanglianghua , thanks for your swift fixing and the link which is really helpful!

happyjin commented 5 years ago

I have found @huanglianghua ExperimentOTB is able to plot the OPE curve by report() function. However, ExperimentVOT is not able to plot it. Will you update this class in order to implement the plot function as well?

huanglianghua commented 5 years ago

@happyjin I had attempted to reproduce the EAO metric and EO curves of VOT experiments, but not successful, they are just too complex :< and the region_overlap used in official VOT toolkit produces slightly different results from standard IoU (see here). We may try to reproduce precise evaluation code for VOT experiments later.

A faster way would be to run your tracker using ExperimentVOT.run method while evaluating the performance using official VOT toolkit. The file structure of saved tracking results from this GOT-10k toolkit is consistent with that from official VOT toolkit.