Absence of refinenet in training scripts

[niqodea]

Hello RIFE authors!

By comparing the original training code v4.0 with the later versions v4.12 and v4.15, it seems like we now no longer train the refinenet, only the flownet. I have some questions about it:

• Is there a specific reason for this?
• Are the refinenet weights ideally independent from the flownet?
• Have you kept the weights of the refinenet fixed from a certain v4.x version onwards?
• Isn't keeping the refinenet in the training forward pass always important, even when the refinenet is not trained? Without it, we are comparing the ground truth with an unprocessed output that was obtained by simply morphing the two before and after images, instead of the output of the entire model pipeline. Am I missing something?

Also, a request: would it be possible to have a version of the latest training code that enables you to also train the refinenet? I assume it would just be a matter of putting back contextnet and unet, correct?

Thank you for your time!

[hzwer]

Hi, we removed refinenet completely. Because we found that although refinenet improves model evaluation indicators, it often performs poorly in real scenarios. The output is obtained by merging two warped images.

[niqodea]

I see, thank you for the quick response! Do you think there might be cases where using refinenet might help improve some interpolations visually, for example in animation? I could see some interpolations benefiting from further postprocessing, since warping the before and after frames might not be enough to achieve some details in the inbetween.