Open danielnapierski opened 1 year ago
Baseline results on standard 8000 test:
[{"test": {"B1": 55.59, "B2": 37.76, "B3": 25.32, "B4": 16.71, "METEOR": 17.56, "ROUGE-L": 38.71, "CIDEr": 48.25, "SPICE": 13.13}}]
vizwiz_joined_captions.json.txt
I split the execution across 10 GPUs by splitting the vizwiz test.json annotation file into 10 files. I ran 10 docker processes (each using a single GPU on 1 of 3 gaia-lg machines). I then combined all the results and submitted one json file.
Team | B1 | B2 | B3 | B4 | ROUGE-L | METEOR | CIDEr | SPICE |
---|---|---|---|---|---|---|---|---|
Unified-io-inference docker | 55.59 | 37.76 | 25.32 | 16.71 | 38.71 | 17.56 | 48.25 | 13.13 |
The baseline captioning implementation would place 9th on the VizWiz Captioning 2021 Leaderboard: https://eval.ai/web/challenges/challenge-page/739/leaderboard/2006
I have run the baseline captioning VizWiz on the standard test 8000 image leaderboard.