search

got-10k / toolkit

Official Python toolkit for generic object tracking benchmark GOT-10k and beyond
http://got-10k.aitestunion.com/
MIT License
554 stars 94 forks source link

Probable issues while evaluating test set On Server #81

Open shash29-dev opened 3 months ago

shash29-dev commented 3 months ago

Hi,

Results are not yet shown on the account. What may be probable issues?

For early submissions, I had some naming differences in the zipped folder than what is expected. I resubmitted the file.. doesn't work. Created a New Tracker and uploaded the correct files with the required structure.. still did not work.

I uploaded SiamFC files, which are provided on the web for reference, and it gave results for it after 10+ Hrs.

I am not sure if there is some error in the submission or if it is some other issue. Can you please share what probable issues submissions have while evaluating on the server? What is the time till we can expect evaluation metrics visible if all is correct, and after how much time do we be sure that results will not come?

In this case, It has been two+ days since I uploaded my results for evaluation on the server.

Xuchen-Li commented 3 months ago

We apologize for the inconvenience you experienced with the website. We are currently attempting to resolve this issue. Thanks for your attention and support for GOT-10k.

Xuchen-Li commented 3 months ago

Thank you once more. As you mentioned in the issue, I have recreated a tracker and submitted the SiamFC example tracking results downloaded from the official website (notwithstanding, we do not recommend users attempt like this). After refreshing the webpage approximately 2 minutes later, I received the evaluation results. I did not encounter any anomalies during the evaluation process using the GOT-10k webpage. The likely reason for the issue is the inconsistency in the format of the tracking results. You can obtain detailed information regarding the tracking result format at http://got-10k.aitestunion.com/submit_instructions.

Xuchen-Li commented 3 months ago

Simultaneously, we are about to upgrade our entire evaluation platform to reduce the occurrence of similar issues. We sincerely apologize for any inconvenience caused to you.

Xuchen-Li commented 3 months ago

To avoid formatting issues with the tracking results, please strictly organize the result file according to the official format and package and submit it using the official toolkit.