search

wilson1yan / VideoGPT

MIT License
962 stars 115 forks source link

Video retrival #4

Closed Tortoise17 closed 3 years ago

Tortoise17 commented 3 years ago

Is this also possible to retrive video using this code like VIDEO BERT? to search in text qwery? and merge frames from database depending on situation of sentence in search.?

Please guide !!

wilson1yan commented 3 years ago

I don't think there's an obvious way to do this without some sort of additional computation / training involved. VideoGPT does not use language as a pre-training modality. You could try learning video representations through generative pre-training on a dataset of videos, and query a dataset for nearest neighbor videos if given (query, video) pairs.