search

hitachinsk / FGT

[ECCV 2022] Flow-Guided Transformer for Video Inpainting
https://hitachinsk.github.io/publication/2022-10-01-Flow-Guided-Transformer-for-Video-Inpainting
MIT License
311 stars 32 forks source link

About image resolution and optical flow resolution #12

Closed 1129ljc closed 2 years ago

1129ljc commented 2 years ago

I noticed that in the code './tool/video_inpainting. Py 'the resolution of the optical flow estimation for low-resolution images is extended, which causes the resolution to not align when the function rf.regionfill() is called. if imgH < 350: flowH, flowW = imgH * 2, imgW * 2 else: flowH, flowW = imgH, imgW After this code is deleted, the sample is generated accurately. Can the author explain the reason for this?v

hitachinsk commented 2 years ago

That may be a bug after refactor, I will check the details. The usage about these two lines is to estimate accurate optical flows. Since we adopt RAFT for flow estimation, we noticed the performance of deep learning-based optical flow estimation methods is related to the frame resolution. If the resolution is too low, the performance of flow estimation will degrade. Therefore, we enlarge the frame resolution when the resolution is too low.

hitachinsk commented 2 years ago

This bug is fixed, I impose the resize operations to optical flows in calculate_flow function in the code ./tool/video_inpainting.py.