search

Mukosame / Zooming-Slow-Mo-CVPR-2020

Fast and Accurate One-Stage Space-Time Video Super-Resolution (accepted in CVPR 2020)
GNU General Public License v3.0
915 stars 164 forks source link

Maintaining FPS while using Super Resolution #14

Closed nelaturuharsha closed 4 years ago

nelaturuharsha commented 4 years ago

Thank you so much for the amazing work and open sourcing it, and actively sorting out queries for the community. There are two queries,

  1. I believe in order to run the model efficiently, rather than loading the images all at once into memory -- one could batch the paths to the images and read them on the fly -- I'll attempt to work on this, might be useful in case of upscaling higher resolution videos.

  2. This might be a stupid question, but I already have a 50 FPS video, and I would like to upscale it using Zooming Slow Mo but maintain the number of frames and not double it, how do I skip the interpolation aspect of the algorithm?

Would appreciate any insights. Thank you, Sree Harsha

Mukosame commented 4 years ago

Hi @SreeHarshaNelaturu , sorry for the late reply! Thanks for your good words!

  1. Now we don't load all images at once during the test phase, as you mentioned. Still, we need to load all the images within on training window (e.g. 1,3,5 for 12345). You can change the window size here to reduce the occupancy of the memory. If you have any ideas on how to make the upscaling more efficient, I'm more than happy to merge you pull request :)

  2. If without the interpolation part, it would be a Video Super Resolution (VSR) task. You can look for the relevant works on it. I'd recommend two projects that are highly relevant to our ZoomingSlowMo: TDAN and EDVR. Both of them are excellent works.

Hope you find my answers helpful :)

nelaturuharsha commented 4 years ago

Thank you so much for the response and I apologize for my delayed reply. It was helpful!