search

hu64 / FFAVOD

Repository for the paper FFAVOD: Feature Fusion Architecture for Video Object Detection
MIT License
24 stars 10 forks source link

Can't reproduce the results #1

Closed abel71 closed 2 years ago

abel71 commented 2 years ago

Hi, I have a hard time reproducing the results reported in your repo/paper using the provided pre-trained models. On the UAVDT test dataset (using the same split you shared /val-1-on-30.json), I attempted to evaluate the shared per-trained model (spotnet2_vid_uavdt.pth), but I was not able to obtain the same mAP as reported in the paper.

image

Is there anything else I could do to reproduce the results.

hu64 commented 2 years ago

Hi, thank you for your interest in our paper! So first thing you should know is that the results reported in the paper are using the official Matlab toolkit provided by the authors. The coco metric is only a hint as to if we're improving are results or not. The main reason they differ (in my opinion) is that the "ignore regions" are automatically handled in the Matlab script. Also, the mAP calculation might be slightly different, I am not sure. Good luck!

Yubzsz commented 2 years ago

Hi, I am currently using the official code CenterNet to train the UA-DTETRAC dataset and have seen that your training accuracy using this method is 83%.But my training accuracy is about 10% worse than yours.I would like to ask you about the config file settings during the training process, such as the learning rate and the selection of the backbone network or other changes.In addition,would you like send me the log txt.Many thanks for your support.

hu64 commented 2 years ago

Hi, are you using the official UA-DETRAC Matlab evaluation tool? Because the results are better with it than the COCO metric. The parameters for my experiments can be found at: https://github.com/hu64/FFAVOD/tree/master/ffavod-experiments. Other values are set to default, which can be found at: https://github.com/hu64/FFAVOD/blob/master/src/lib/opts.py

Malikanhar commented 2 years ago

Hi @Yubzsz are you able to sign up and login to UA-DETRAC website? I always getting Whoops, looks like something went wrong. while signing up on the website. And if you don't mind, could you share the UA-DETRAC annotations and its evaluation tools for me? Thanks in advance

Yubzsz commented 2 years ago

Hi @Malikanhar I download from https://pan.baidu.com/s/130w_33XEnwTjP3OOqdkfHA，Extraction code:4rgd. I didn't have the evaluation tools either.

Yubzsz commented 2 years ago

Thank you very much! @hu64 I use the hourglass to train,and achieve the same effect.