search

ttengwang / PDVC

End-to-End Dense Video Captioning with Parallel Decoding (ICCV 2021)
MIT License
207 stars 23 forks source link

caption my custom video #2

Closed dawnlh closed 2 years ago

dawnlh commented 3 years ago

Hi @ttengwang ~ Thanks for the sharing of your wonderful work! I want to caption my custom video, but unfortunately I find that most codes for captioning are starting from extracted features, and little instructions are provided for the extraction process. It's very inconvenient for me, cause I'm not so familiar with the captioning task, and I just want to utilize the tool for some applications. So could you please kindly give me some detailed instructions on how I can get the captions from a raw video? I will appreciate it a lot!

Thanks, Zhihong

ttengwang commented 3 years ago

Hi @dawnlh , thanks for your interest in our work.

This repo BMT (https://github.com/v-iashin/BMT#details-on-feature-extraction) can deal with custom videos, it may meet your need.

For our paper, we directly download the C3D and TSN features from other repos rather than extract them by ourselves. The TSN features are provided by densecap using this repo. C3D features are provided by ActivityNet Challenge (no code). I3D+VGGish features are from BMT. You can find the instructions in the above repos.

I noticed the inconvenience for industrial applications and plan to add the feature extraction part in my spare time, but it won't be done very soon.

dawnlh commented 3 years ago

Thanks a lot for your detailed advice 🤝 . BMT does provide detailed instructions on how to deal with custom videos, but unfortunately, my video has no audio information, which BMT needs. I will try to figure out this problem~ And also, looking forward to the feature extraction part for PDVC, and thanks for your effort to make it convenient for real applications!

ttengwang commented 2 years ago

@dawnlh Hi, now this repo supports captioning custom videos.

dawnlh commented 2 years ago

@dawnlh Hi, now this repo supports captioning custom videos. Wonderful！ Much thanks for your effort and kind reminder!