How did you learn IFNet_m?

[JingangHuh]

Thanks for providing the code.

I wonder how I learned IFNet_m. First, let me tell you how I learned the network.

The first method was learned from scratch using vimeo_septuplet. However, since the forward function of IFNet_m receives the timestep as a batch, I modified it to change the batch of timestep value to 1 channel frame in the learning phase. But learning is not working properly.

The second method is to train at timestep 0.5 with vimeo-triplet, and fine-tuning this model with vimeo-septuplet. But still not learning properly.

In the video below, you can see the result of the network I learned. If you look closely, only timestep 0.5 is normal, and 0.25 and 0.75 overlap slightly.

<HD720p_GT/parkrun_1280x720_50/frame 5> - timestep 0.25

<HD720p_GT/parkrun_1280x720_50/frame 6> - timestep 0.5

<HD720p_GT/parkrun_1280x720_50/frame 7> - timestep 0.75

I would appreciate it if you could tell me how you learned IFNet_m. Or if you point out my problems, I'd appreciate it.

Thanks, I'll wait for your reply.

[hzwer]

Hi, I train RIFEm from scratch using vimeo_septuplet. The related code is recently released on https://github.com/hzwer/arXiv2021-RIFE/blob/168b9d120ef65a25a166623a01b8fee4420da7e0/dataset.py#L108. And this code is checked during #238. A common problem is not adjusting the timestep when swapping I0, I2 (data augmentation).

[JingangHuh]

I really appreciate for your reply. My problem was that I didn't change the timestep when swapping I0 and I2. I solved it by changing it. thanks again.

[hzwer]

Good luck. I will close this issue.