search

jayleicn / singularity

[ACL 2023] Official PyTorch code for Singularity model in "Revealing Single Frame Bias for Video-and-Language Learning"
https://arxiv.org/abs/2206.03428
MIT License
130 stars 14 forks source link

Apply to new dataset #11

Closed nikky4D closed 2 years ago

nikky4D commented 2 years ago

Hi, Can you give me instructions of how to set up for my own custom dataset? I have about 100 folders of images, with each folder containing the images from a video clip. I also have captions for each of these video clips. How would I use this custom dataset for finetuning for text-video retrieval?

jayleicn commented 2 years ago

Hi @nikky4D, I have updated the readme to include instructions on custom datasets, please see https://github.com/jayleicn/singularity#pre-training-on-custom-data.

nikky4D commented 2 years ago

Thanks. Are these instructions the same for finetuning? My dataset is too small for full pretrain so I want to attempt finetune to see the results.

jayleicn commented 2 years ago

Yes.

nikky4D commented 2 years ago

Thank you