baowenbo
commented
5 years ago Yes, I do find these problems. I think the abnormal blocks are caused by the high resolution of images. You can also check out my supplementary file. Here, I quote our discussion for this problem below:
Although we show in the main paper that our DAIN model achieves better PSNR and SSIM values against the MEMC-Net [2] algorithm on the HD dataset, we observe that there are some annoying artifacts in the Bluesky and Sunflower videos. Specifically, we discover that these artifacts are introduced by the frame synthesis network. In Figure 14, we present the 4-th frame results of the Bluesky video. The three images are generated by the adaptive warping layer, the frame synthesis network and the corresponding ground-truth frame respectively. The artifacts appeared in the flat sky area of the synthesized result suggest us that a more robust network should be proposed to deal with high-resolution images.
I think plenty of problems such as high resolution, large displacement, need further analysis in the video frame interpolation task as in other tasks such as optical flow. For high resolution images and large displacements, the receptive field of our network may still be insufficient. Possibly, the scale-recurrent network would be a good idea to try.
Hi, @baowenbo ，Thank you for your great work and sharing the code. The test results were really surprising, but I found some blurry or not-expected results in DAIN_HD_videos as shown below. Can you tell why do these happen or some ideas for improvement?
some abnormal blocks in interpolated frames
blurry when large displacement appeared