search

CTU-IIG / kcf

Kernelized Correlation Filter tracker
13 stars 6 forks source link

Low accuracy for ball1/default/opencvfft-st #31

Open wentasah opened 6 years ago

wentasah commented 6 years ago

When running make test TESTSEQ=ball1 BUILDS="fftw opencvfft-st", the following results are produced:

! ball1  default  fftw          Average processing speed: 155.54ms (6.43 fps)   Average accuracy: 0.75 >= 0.70 .... ok
! ball1  default  opencvfft-st  Average processing speed: 101.03ms (9.90 fps)   Average accuracy: 0.36 >= 0.70 .... ACCURACY
! ball1  fit128   fftw          Average processing speed: 122.32ms (8.18 fps)   Average accuracy: 0.78 >= 0.70 .... ok
! ball1  fit128   opencvfft-st  Average processing speed: 184.78ms (5.41 fps)   Average accuracy: 0.78 >= 0.70 .... ok
Shanigen commented 6 years ago

I tried to look for the problem and ran the tracker multiple times, but every time the tracker passed the test. I know that the current version has the threshold set to 0.35 but all of the accuracies were over 0.70 during my testing. The output is here: 

! ball1  default  fftw          Average processing speed: 26.51ms (37.72 fps)   Average accuracy: 0.75 >= 0.35 .... ok
! ball1  default  opencvfft-st  Average processing speed: 83.28ms (12.01 fps)   Average accuracy: 0.75 >= 0.35 .... ok
! ball1  fit128   fftw          Average processing speed: 86.56ms (11.55 fps)   Average accuracy: 0.77 >= 0.35 .... ok
! ball1  fit128   opencvfft-st  Average processing speed: 109.24ms (9.15 fps)   Average accuracy: 0.78 >= 0.35 .... ok
wentasah commented 6 years ago

Hmm, it seems it depends on OpenCV version. Which version do you have?

wentasah commented 6 years ago

Hmm, this is what I get with my OpenCV 3.4:

! ball1  default  fftw           Average processing speed: 29.07ms (34.40 fps)   Average accuracy: 0.36 >= 0.35 ... ok
! ball1  default  opencvfft-st,  Average processing speed: 32.54ms (30.73 fps)   Average accuracy: 0.36 >= 0.35 ... ok
! ball1  fit128   fftw           Average processing speed: 44.05ms (22.70 fps)   Average accuracy: 0.78 >= 0.35 ... ok
! ball1  fit128   opencvfft-st,  Average processing speed: 49.61ms (20.16 fps)   Average accuracy: 0.78 >= 0.35 ... ok

Even fftw has lower accuracy. Can you try it with different opencv versions/builds? It could be a problem of OpenCV, but maybe we have a bug in the code.