search

hzwer / Practical-RIFE

More practical frame interpolation approach.
MIT License
631 stars 69 forks source link

Will there be another Rife model? #68

Open netExtra opened 6 months ago

netExtra commented 6 months ago

Rife v4.15 and 4.15 lite are very good but still has some issues.Rife v4.16 lite is not quite as good as v4.15. Will there be another Rife model?

hzwer commented 6 months ago

Hi, I haven't done any related experiments for a while. I'll see if I can start doing some more experiments this week. Recently I'm developing models for stepchat.cn :)

netExtra commented 6 months ago

Hi, I haven't done any related experiments for a while. I'll see if I can start doing some more experiments this week. Recently I'm developing models for stepchat.cn :)

Thanks. SVP is improving Rife SC which helps but there's still problems with certain types of movement as shown in my previous issue here: https://github.com/hzwer/Practical-RIFE/issues/34#issuecomment-1747789930

Nick088Official commented 6 months ago

I hope there will be a new one soon, and maybe a deeper guide on all the possible arguments

netExtra commented 6 months ago

I hope there will be a new one soon, and maybe a deeper guide on all the possible arguments

Yeah I hope so too. It would be nice to have another one build upon the v4.15 model further by reducing artefacts :)

netExtra commented 6 months ago

@hzwer Sorry to bug you but will you have time to produce a new Rife model this week? I hope so. If not then thanks anyway :)

hzwer commented 6 months ago

@netExtra 4.17 is released, not sure if it will be better.

netExtra commented 6 months ago

@netExtra 4.17 is released, not sure if it will be better.

Thanks for this. From what I can see it seems to be near identical to v4.15. But I do appreciate the new release đź‘Ť

BTW You were originally going to make v4.16 lite the baseline so were you trying to make it as good as v4.15 but using less resources? Do you think this is as good as Rife can get or is there a possibility of the model getting better?

Update: I found a case where v4.15 is still a little better than v4.17 when using a modified Scene Detection algorithm so it looks like overall v4.15 is still the best.

Thanks again :)

hzwer commented 6 months ago

For 4.16.lite, I'm worried that the space for model compression is already very limited. Another possible solution is to add data, I am not sure when this work will start.

v4.17 seems to bring a very different visual effect to anime scenes. I don’t know whether it is generally better or not.

netExtra commented 6 months ago

For 4.16.lite, I'm worried that the space for model compression is already very limited. Another possible solution is to add data, I am not sure when this work will start.

v4.17 seems to bring a very different visual effect to anime scenes. I don’t know whether it is generally better or not.

Thanks. But many if not most of us use this for live action, not Anime. We have been lucky that some of the model improvements also helped with live action but it would be nice if a better model for live action was possible.

mn7216 commented 6 months ago

For 4.16.lite, I'm worried that the space for model compression is already very limited. Another possible solution is to add data, I am not sure when this work will start. v4.17 seems to bring a very different visual effect to anime scenes. I don’t know whether it is generally better or not.

Thanks. But many if not most of us use this for live action, not Anime. We have been lucky that some of the model improvements also helped with live action but it would be nice if a better model for live action was possible.

RIFE's training code is freely available. You can always train a model for live action if you would like. Be grateful that hzwer is still working on RIFE after all this time, which the large majority of model authors don't do. No need to continuously bug him about releasing new models

hzwer commented 6 months ago

@mn7216 Don't worry. In fact, having people paying attention is a great source of motivation. 4.17 is exactly an experiment I decided to do after seeing this issue :) The main thing stopping me from updating is that I really don't have any better ideas...

netExtra commented 6 months ago

For 4.16.lite, I'm worried that the space for model compression is already very limited. Another possible solution is to add data, I am not sure when this work will start. v4.17 seems to bring a very different visual effect to anime scenes. I don’t know whether it is generally better or not.

Thanks. But many if not most of us use this for live action, not Anime. We have been lucky that some of the model improvements also helped with live action but it would be nice if a better model for live action was possible.

RIFE's training code is freely available. You can always train a model for live action if you would like. Be grateful that hzwer is still working on RIFE after all this time, which the large majority of model authors don't do. No need to continuously bug him about releasing new models

I have a number of scenes I use to test Rife and I've watched @hzwer's models get better and better over time with less artefacts. We seem to have reached da plateau with v4.15. So would it be possible for me to use those scenes to train v4.15? If it is possible then how because the instructions are not clear to me.