search

microsoft / SwinBERT

Research code for CVPR 2022 paper "SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning"
https://arxiv.org/abs/2111.13196
MIT License
238 stars 34 forks source link

Missing Caption Files For YouCook2 Dataset #50

Open fzohra opened 1 year ago

fzohra commented 1 year ago

Hi,

I am unable to evaluate SwinBert on the YouCook2 dataset due to missing coco captions files. The evaluation expects the following files:

validation.caption_coco_format.json
testing.caption_coco_format.json
training.caption_coco_format.json

However, the annotations downloaded for YouCook2 using the provided scripts does not contain these files.

I'm training/evaluating with the default configurations provided in the source code and README so I expect I should be able to run the code without modifying the evaluation logic for this dataset. Please advise on how to train/evaluate this dataset. Thanks!

AdrienneDeganutti commented 11 months ago

Hi @fzohra,

I was able to generate the _.caption_coco_format.json files using the tsv_preproc_youcook2.py pre-processing script.

I have found that this will not generate a file for the testing split as these are processed from the dataset's ground truth annotations: youcookii_annotations_trainval.json which only include training and validation splits.

Hope this helps!