Question about the semi-automatic dataset creation process

[ooza]

Hello, Thanks a lot for making available this amazing work! I'm interested in the semi-automatic dataset creation. Any useful detail about this framework will be much appreciated. The script generate_instruction_qa_semi_automatic.py requires

• the Path to the ground truth captions file. but what if I don't have gt captions, I mean only raw videos, can I still use your framework? How?
• the Dir path to off-the-shelf model predictions, can you tell me how create these predictions ?

It will be useful to provide a running example of this script python generate_instruction_qa_semi_automatic.py ----gt_caption_file ... --pred_dir ... ?

[mmaaz60]

Hi @ooza,

I appreciate your interest in our work. We recently released our work called VideoGPT+ and an improved semi-automatic video annotation pipeline. All the scripts to run the pipeline are also released.

Please check it out at GitHub, HuggingFace.

Please let me know if you have any questions. Good Luck!