Rife 4.26 Anime deformity decreased but inbetween frames are less visible

[2994dd42d75b0]

I removed static frames from Sample Anime video and tested all few versions

Original video 6fps

https://github.com/user-attachments/assets/d5decd92-1b83-4251-89f2-d0f289f6ae76

4.26

https://github.com/user-attachments/assets/e2edef06-f30b-49b3-b28b-84555274038f

Union model from GMFSS_Fortuna Got best result

https://github.com/user-attachments/assets/9e66c123-72d8-458b-b4f0-87f54d4700f6

Any way train rife with GMFSS model ?

[hzwer]

Okay, I will spend time to understand this project.

[1kartafal4]

@hzwer Are you one of the developers? RIFE: Real-Time Intermediate Flow Estimation...

So I don't know how this gmffs model scores objectively in PSNR, SSIM etc. but holy moly! Subjectively I perceive this GMFSS model much, much better than the rife 4.25 (lite) & 4.26 model (which are the best at the moment for real video and anime/cartoon). I usually take my subjective tests a bit more serious as mentioned here, especially when it comes to low playback speed, time, and comparisons. Normally, I see people doing subjective tests in a much flawed manner, and thus the conclusions are a fallacy and more useless.

So in x0.125 & x0.25 playback speed, were most artefacts are visible and do not escape our flawed eye/brain perception, it's clear that this model has it's own weaknesses, but even at such low playback speed, I see it having more advantages than disadvantages. It's a clear winner.

Basically, comparison at x0.125 & x0.25 playback speed: Rife 4.26: Disadvantages are, the model has clearly more ghosting (trailing is the correct term?) and warping when it comes to objects of all sizes or frame transitions. Also more deformation, but that's not always a clear case; sometimes it's worse, sometimes better. In general also more stuttery. The advantage I see there is less "blending in" aka blurred, between objects (not sure how to phrase it correctly), which leads to the advantage that some objects, lines etc. are remain more detailed/sharp. But this is not always the case.

Now with GMFSS at x0.125 & x0.25 playback speed: I see it's the exact opposite; that means the only disadvantage I see more blending in of objects, lines etc. but the advantages much outweigh the disadvantages.

Maybe this image makes it more clear. (RIFE) The boots, hair clothing, arm, have the usual ghosting like double images and warping. GMFSS model remain more detailed here due to having almost none of that, which migh contradict what I wrote before. it's not always the case. The starting scene with the dragon, the purple blobs inside, remain more clear with RIFE.

At x0.5 playback speed the differences become more clearer, as the discrepancy between each advantages/disadvantages. As linked above, our flawed human perception can't keep up processing all the information at higher framerate, and thus the disadvantages of RIFE (deformation, warping), become more noticeable. It is especially the stuttering-like aspect of RIFE, which becomes much more clear now. The more blending in aka blurry of GMFSS is not that noticeable as before

At x1 (normal) playback speed, GMFSS looks much better. The disadvantages of the RIFE have a much bigger impact here. Especially the stuttering - RIFE model basically looks like a stutter-show to me here.

[1kartafal4]

Here is some other comparison, stopped perfectly at 00:06:265 time for both. The RIFE model is still all ghosting/trailing and warping-like. GMFSS has the mentioned blend in/blur, which can be seen at the right foot.

Here some other comparison to make it clear what I mean; stopped at 00:01:097 RIFE model remains more cleared (detailed/sharp) as mentioned above, but at the cost of much more ghosting/trailing, warping, deformation. GMFSS is the exact opposite. It's all like blended in, blurred together.

[1kartafal4]

As mentioned, all those disadvantages of RIFE model become more noticeably at normal playback speed (framerate), and it looks a lot stuttery. As framerate increases, I notice the disadvantage of GMFSS (blend in, blur) less and less. The advantage being much more fluid keeps increasing.

And here the framerate is only 30 fps. That means: For 60 fps - were I assume most people like me will use model for playback -, GMFSS should be perceived even better.

Tl;dr I pick GMFSS model over RIFE. Maybe this subjective comparisons helps for your training of the models. If yes, feel free to link it to them. ;)