search

microsoft / UniVL

An official implementation for " UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation"
https://arxiv.org/abs/2002.06353
MIT License
335 stars 54 forks source link

How to only input text feature or video feature #40

Open tingchihc opened 2 years ago

tingchihc commented 2 years ago

I want to only input text feature or video feature in UniVL. In this paper, it said that one transformer combines text representation T and video representation V. Could you tell me how to change it to only input T or V into UniVL? thanks

ArrowLuo commented 2 years ago

Hi @ting-chih, sorry for the delayed reply. The model will also need T and V, which can be masked if you need only to input one of them. For example, for only V, T is [CLS][SEP], and for only T, V is all zero. Best~

tiesanguaixia commented 1 year ago

I want to only input text feature or video feature in UniVL. In this paper, it said that one transformer combines text representation T and video representation V. Could you tell me how to change it to only input T or V into UniVL? thanks

Hi! Do you know how to download the raw videos of YouCook2? Thank you very much!