search

votchallenge / toolkit

The official VOT Challenge evaluation and analysis toolkit
http://www.votchallenge.net/
GNU General Public License v3.0
153 stars 43 forks source link

Parallel tracker evaluation #52

Open davidyang180 opened 2 years ago

davidyang180 commented 2 years ago

Hello! We participated in the VOT challenge for the first time, and initially tried the testing process of the toolkit. It seems that we can only test in a single thread; are we only able to evaluate through the official VOT toolkit?Can we do multi-threaded testing on the basis of VOT toolkit?

lukacu commented 2 years ago

At the moment we only have single-threaded testing as multi-threaded tests can impact the overall performance in uncertain ways. Still, I know that this would be desired and it can be added (I will see how much time will I have in the upcoming weeks).

lukacu commented 1 year ago

Well, I am not sure if you can complete all the experiments this way, pytracking does not have realtime challenge implemented. All experiments must be complete to submit to the challenge. But there is plenty of time left so you can easily test with pytracking and then finally with vot-toolkit.

On Thu, Mar 31, 2022 at 12:17 PM davidyang @.***> wrote:

At the moment we only have single-threaded testing as multi-threaded tests can impact the overall performance in uncertain ways. Still, I know that this would be desired and it can be added (I will see how much time will I have in the upcoming weeks).

Hello! Recently, I tested a multi-threaded test program for the VOT2022-STB public data set locally through the pytracking toolkit, but after evaluating the test results through the VOT toolkit, I found that there is a certain error compared with the results running in the VOT toolkit. After comparing the two result output files, except for a slight difference in decimal places of 3-6 orders of magnitude, the output results are basically the same, but the EAO of the evaluation results differs by 0.5-1 percentage points; What is the reason for this, I don't think the difference after 3 decimal places will cause such a big difference in the evaluation results.

— Reply to this email directly, view it on GitHub https://github.com/votchallenge/toolkit/issues/52#issuecomment-1084374707, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGVY6MHWDBL4CZNFBABIALVCV3SZANCNFSM5RZTK7KA . You are receiving this because you commented.Message ID: @.***>

-- Luka Čehovin Zajc http://luka.tnode.com