search

chaiNNer-org / spandrel

Spandrel gives your project support for various PyTorch architectures meant for AI Super-Resolution, restoration, and inpainting. Based on the model support implemented in chaiNNer.
MIT License
103 stars 7 forks source link

EMA-VFI interpolation and RVRT #228

Open zelenooki87 opened 2 months ago

zelenooki87 commented 2 months ago

Hi,

I noticed you've begun implementing support for video super-resolution (VRT) and FLAVR. When can we expect these features to be officially "working" in the Chainer framework?

I also have a suggestion: consider adding RVRT as a potential architecture. It's by the same author as VRT but less computationally expensive. https://github.com/JingyunLiang/RVRT

One more request: EMA-VFI interpolation is one of the best open sourced, interpolation methods available right now. Could you please consider including it as well? https://github.com/MCG-NJU/EMA-VFI

Thanks!

RunDevelopment commented 2 months ago

same author as VRT but less computationally expensive.

And more complex architecturally :)

I plan to add support for a lot more video SR and frame interpolation architectures in the future. This first PR just includes a few architectures, so I can figure out how to build the necessary infrastructure.

zelenooki87 commented 2 months ago

This sounds great! By the way, I noticed that the safmn GitHub repository has been updated with new models. Could you update the safmn architecture in Spandrel to support them? Thanks! https://github.com/sunny2109/SAFMN

joeyballentine commented 2 months ago

When can we expect these features to be officially "working" in the Chainer framework?

Potentially soon. We'd just have to implement them in a not very ideal way to work around chaiNNer's limitations. If we wanted to do it correctly, then it would take much much longer

zelenooki87 commented 2 months ago

To avoid opening a new issue, I'm writing here. The author of SRFormer released a real-world model. For some reason, I'm getting patch artifacts on the images. The inference code for the model seems to have changed, which is likely the cause. https://github.com/HVision-NKU/SRFormer The specific model I'm referring to is SRFormer_S_RealSR.pth. ( https://drive.google.com/drive/folders/1D5ER_HwYJoyZCcrKVstwE-iEl0hXulwd ) If possible, could you please check the inference code? I can attach screenshots of the patch artifacts if needed. All other SRFormer models work correctly. Thanks.

RunDevelopment commented 2 months ago

@zelenooki87 Please don't reuse issues for unrelated topics. SRFormer has nothing to do with this issue. We use GH issues to track bugs and progress, so each issue should only cover one thing. I made a new issue for SRFormer for now.