search

Mukosame / Zooming-Slow-Mo-CVPR-2020

Fast and Accurate One-Stage Space-Time Video Super-Resolution (accepted in CVPR 2020)
GNU General Public License v3.0
908 stars 165 forks source link

why DAIN before EDVR #35

Closed oliverxudd closed 4 years ago

oliverxudd commented 4 years ago

Hi Musosame, congrats on your achivement on this new task!

I have a question with regards to the order of the VFI and VSR method. In your paper, DAIN with EDVR make a strong competitor for Zooming SlowMo. But why video frame interpolation before video super resolution? Intuitively, enhanced spatial resoltion will improve optical flow estimation, and then frame interpolation too. But not opposite. have you conduct experiment considering this factor? the order of VFI and VSR?

Mukosame commented 4 years ago

Hi @oliverxudd , this is really a good question.

The main reason is to reduce computation. If we first perform SR before VFI, VFI will work on an HR space, which greatly increases the computation time. For the upscaling factor: 4, a VFI model running on HR frames will become around 16 times slower, not to mention the high occupation of GPU memory.

oliverxudd commented 4 years ago

OK, that makes sense. thank you for your reply.