search

SoccerNet / sn-tracking

Repository containing all necessary codes to get started on the SoccerNet Tracking challenge. This repository also contains benchmark methods to get started.
71 stars 17 forks source link

SoccerNet-Tracking Challenge 2023 Submission Evaluation Issue #18

Open Seattle1917 opened 5 months ago

Seattle1917 commented 5 months ago

Hello Author,

I am currently preparing to conduct tests on the Challenge2023 dataset (including SNMOT-0, ..., SNMOT-096). In accordance with the instructions on the official website, I have prepared result files named SNMOT-XXX.txt, with each line formatted as [frame ID, track ID, Upper left coordinates (x,y), width, height, confidence score, -1, -1, -1]. Additionally, I have compressed all sequence results into a zip file called soccernet_mot_results.zip. However, when I try to submit my results on the website, I encounter the following error:

Traceback (most recent call last): File "/code/scripts/workers/submission_worker.py", line 538, in run_submission submission_metadata=submission_serializer.data, File "/tmp/tmpoe_6yy6c/compute/challenge_data/challenge_1951/main.py", line 62, in evaluate test_annotation_file, user_submission_file, split=split) File "/usr/local/lib/python3.7/site-packages/SoccerNet/Evaluation/Tracking.py", line 120, in evaluate dataset_list = [trackeval.datasets.MotChallenge2DBox(dataset_config)] File "/usr/local/lib/python3.7/site-packages/trackeval/datasets/mot_challenge_2d_box.py", line 126, in init curr_file)) trackeval.utils.TrackEvalException: Tracker file not found: test/data/SNMOT-001.txt

Is it a problem with my file format? How should I adjust it to resolve this issue? I am looking forward to your reply and am very grateful for your assistance!

SilvioGiancola commented 5 months ago

Hi, are you not confusing the test set and the challenge set? I would recommend you check locally if the results on the test set are in the right format (that you don't have an extra folder on your zip, etc...) before submitting it on eval.ai.

Seattle1917 commented 5 months ago

Hello, Thank you very much for your response. The results generated by our method have been verified on our local test set. We are now using the same method to experiment on the Challenge dataset, and the resulting files have the same format. I have compressed the files SNMOT-021.txt, ..., SNMOT-096.txt together into a single file named soccernet_mot_results.zip, ensuring that there are no additional files in the zip archive besides the SNMOT-XXX.txt result files. After that, I went to the submission page at https://eval.ai/web/challenges/challenge-page/1951/submission. I first selected the "Phase: Challenge Set" option and then chose the "Upload file" type to make the submission. Unfortunately, when I checked my submission results, it showed the issue mentioned earlier (status: Failed). However, when I used our method to generate results on the test set and selected the "Phase: Test Set" option, choosing the "Upload file" type to make the submission, it was successful (status: Finished). This is quite perplexing, and I am unable to resolve it. I hope to receive your assistance.