votchallenge / toolkit

The official VOT Challenge evaluation and analysis toolkit
http://www.votchallenge.net/
GNU General Public License v3.0
153 stars 43 forks source link

About VOT2022 sequestered dataset submit singularity container to run vot evaluation command #58

Closed davidyang180 closed 2 years ago

davidyang180 commented 2 years ago

Hi! We currently need to submit a singularity container for evaluation on the VOT2022 sequestered dataset, but because the VOT toolkit will write the track results in the file system during evaluation and analysis, and the sif container is a read-only system, so the sif container maybe cannot be used for run. The official documentation of the singularity container says that the read-only sif container needs to be converted to a sandbox writable container. We found that using the singularity command: singularity build --sandbox --fakeroot xxx/ xxx.sif

Singularity>vot evaluate --workspace <workspace-path> <tracker-name>
Singularity>vot analysis --workspace <workspace-path> <tracker-name> --format json

To convert the sif container into a writable sanbox can run normally. Does the official approve this submission method? By the way, does the submission of VOT2022 in the sequestered dataset need to submit the sif container? Or def definition files as well as source code? Because the capacity of the sif container is relatively large, it may not be submitted by email. Is it allowed to provide a Google drive link.