search

haltakov / natural-language-youtube-search

Search inside YouTube videos using natural language
MIT License
912 stars 72 forks source link

show seconds #5

Closed roloydman closed 3 years ago

roloydman commented 3 years ago

Hi. Would it be possible to show in which second each of the three frames was found? Don't know anything about programming but think it would be useful for the application. Congratulations for the work.

haltakov commented 3 years ago

This should be quit easy to do! I'll implement it tomorrow and let you know

roloydman commented 3 years ago

thanks man. this will help me with video editing.

jackylee1 commented 3 years ago

well done,i have done the same job recently. i want to know whether you store the extracted image on the google drive,if no,how can i search when i do this next time. and the colab can only work for sometime,what if i want to deploy on my computer,does it need high-efficiency computer?

jackylee1 commented 3 years ago

further,any progress on training my own data model,cause sometimes i need more accurate result on specific man or things?

haltakov commented 3 years ago

@roloydman Sorry for my late response... I added the time in the video now and also a link jumping directly to that time. I hope this helps you.

haltakov commented 3 years ago

@jackylee1 - the images are not stored on Google Drive, but only in memory (video_frames). If you want to have this permanent, you could store the images as files and also the precomputed CLIP feature vectors. After that you just need to load them. I did something similar in my other project: https://github.com/haltakov/natural-language-image-search. I think you managed to do it there right?

You can deploy the same code on your own PC as well. CLIP actually works quite well on the CPU as well, but the computation of the feature vectors will be slower than on Colab. When you are doing pip install torch you need to choose the right version for your system - refer to this page: https://pytorch.org/get-started/locally/

Regarding training... this is much more difficult to do. CLIP was trained on a huge dataset and the training code is not open. There are some open source implementations, but it will not be easy and will require significant GPU resources. OpenAI promised, however, to release a bigger CLIP model, which should have better performance.

roloydman commented 3 years ago

much better than i imagined. thanks again

jackylee1 commented 3 years ago

thanks for your response