How to store large amounts of test annotation data?

[davnov134]

Hello,

we are trying to host a challenge which would compare user's submissions with ground truth image data. We have roughly 10GB of such ground truth data which needs to sit on the evaluation server. It is not obvious where to put these files, I assume that pushing those into the github repository is not the option here?

Thanks!

[davnov134]

Thank for the eval_ai team for following up via email.

The solution is:

1. Upload the sample annotations for the phases to github. (This is required as without annotations, the challenge can't be created.)
2. Once you create the challenge, install EvalAI cli tool: pip install evalai
3. Use the command -- evalai challenge 1 phase 4 submit --file annotation.json --large --annotation (where 1 and 4 are sample challenge and phase ids) to upload a large annotation file.

In my case, there are many images and not a single annotation file, so we packed all images to an hdf5 file and uploaded the file.

[RishabhJain2018]

Hey @davnov134 , Thank you for putting up the solution here.