About RFR module

[helloimmelie]

Hello,

I think there is a typo in the link below.

https://github.com/lisiyao21/AnimeInterp/blob/main/models/rfr_model/rfr_new.py#L127

The variable might be declared as f12_init.

And I have a question about RFR module . According to Section 4.2, coarse flow f_0->1 should be multiplied with exp{-g^2}. But I couldn't find this at the code. Could you tell me where this part exist in the code?

Thank you.

[helloimmelie]

And I have another question,

To verify the EPE, I loaded the RFR checkpoint, but model structure is different from which described in rfr_new.py

So I modified the model structure along the saved in the checkpoint, and I tested the model through test_anime_sequence_one_by_one.py, but metrics are lower than before I modifed the model structure.

I am confused, which RFR structure should I refer?

[lisiyao21]

And I have another question,

To verify the EPE, I loaded the RFR checkpoint, but model structure is different from which described in rfr_new.py

So I modified the model structure along the saved in the checkpoint, and I tested the model through test_anime_sequence_one_by_one.py, but metrics are lower than before I modifed the model structure.

I am confused, which RFR structure should I refer?

Thanks for watching the code in detail! I will double check the rfr weights I shared you, and reply you in detail after ddl of CVPR submission.

[helloimmelie]

Hello, any updates on this issue?

[lisiyao21]

Hello, any updates on this issue?

Hi Hi. I've just submitted the CVPR supp files.

As to the first question, it seems you are right that the network did not use the 3-layer mask due to the typo. We are checking the former versions of code. However, the benchmark scores reported in the paper is absolutely confirmed to be the same as released model.

As to the second question, since I don't know which part you've modified, I can't give much comments. But, PSNR score of 14+ is extremely low that means the networks doesn't do anything right. I suppose that you add some layers back according to the pretrained weights of RFR. Please do not do so, because there may be some layers (e.g., normalization layers) that we declared but did not use. Indeed, we tried several versions during training the models, and we don't delete those declarations then but just commented the relative lines in forward function. So it is highly risky to "restore" the structures via the weights. (If I guess wrongly please leave further comments~)

In all, please prevail the released code and weight.

[lisiyao21]

Hello, any updates on this issue?

And if you think the github communications are too slow you could indicate a better one (:-3> <)

[helloimmelie]

Hello, Thank you for your reply. I will update the issue if I found another one!

[lisiyao21]

Hello, Thank you for your reply. I will update the issue if I found another one!

You are very welcome! :-)