search

SysCV / shift-detection-tta

This repository implements continuous test-time adaptation algorithms for object detection on the SHIFT dataset.
MIT License
18 stars 2 forks source link

No results for the submition VIS Eval server. #13

Closed YuyangSunshine closed 5 months ago

YuyangSunshine commented 10 months ago

Hi~ I have submitted the zip file following the submission guidelines, but still have no evaluation results. Is it because of the format of my submission file, or what is the reason? image

YuyangSunshine commented 10 months ago

For now, the submission is been cancelled. And the Stderr file shows:

Traceback (most recent call last): File "/code/scripts/workers/submission_worker.py", line 514, in run_submission submission_output = EVALUATION_SCRIPTS[challenge_id].evaluate( AttributeError: module 'challenge_data.challenge_6' has no attribute 'evaluate'

suniique commented 10 months ago

Hey @YuyangSunshine, thanks for your question! Could you please upload the file once again? Lately, our servers have been quite busy, which sometimes causes worker restarts. As a result, pending tasks might get interrupted. If your submission has been pending for too long (a couple of hours) kindly resubmit. Apologies for any inconvenience caused!

YuyangSunshine commented 10 months ago

I uploaded again. But there is a new keyerror:

Traceback (most recent call last): File "/code/scripts/workers/submission_worker.py", line 514, in run_submission submission_output = EVALUATION_SCRIPTS[challenge_id].evaluate( File "/tmp/tmp2422svq5/compute/challenge_data/challenge_6/main.py", line 181, in evaluate result_dict = evaluate_shift( File "/tmp/tmp2422svq5/compute/challenge_data/challenge_6/main.py", line 117, in evaluate_shift result_dict["overall"] = result_dict["mAP"] - 2 * result_dict["mAP_drop"] KeyError: 'mAP'