search

chaiNNer-org / spandrel

Spandrel gives your project support for various PyTorch architectures meant for AI Super-Resolution, restoration, and inpainting. Based on the model support implemented in chaiNNer.
MIT License
103 stars 7 forks source link

SRFormer_S_RealSR doesn't work #236

Closed RunDevelopment closed 2 months ago

RunDevelopment commented 2 months ago

Original report by @zelenooki87:

To avoid opening a new issue, I'm writing here. The author of SRFormer released a real-world model. For some reason, I'm getting patch artifacts on the images. The inference code for the model seems to have changed, which is likely the cause. https://github.com/HVision-NKU/SRFormer The specific model I'm referring to is SRFormer_S_RealSR.pth. ( https://drive.google.com/drive/folders/1D5ER_HwYJoyZCcrKVstwE-iEl0hXulwd ) If possible, could you please check the inference code? I can attach screenshots of the patch artifacts if needed. All other SRFormer models work correctly. Thanks.

RunDevelopment commented 2 months ago

The inference code for the model seems to have changed, which is likely the cause.

We already use the new code.

This means that those problems are with the model and/or architecture itself, so there's nothing we can do about it. Sorry @zelenooki87.

zelenooki87 commented 2 months ago

The inference code for the model seems to have changed, which is likely the cause.

We already use the new code.

This means that those problems are with the model and/or architecture itself, so there's nothing we can do about it. Sorry @zelenooki87.

The model I'm interested in is actually "SRFormer_S_RealSR.pth." While SRFormer_realworld_W22C180.pth seems to be working well, these models were trained differently. If it's not a hassle, could you double-check? I'm happy to upload some examples for you to examine.

https://mega.nz/folder/cwRgXbKL#SnVHDDCTP-VYbtyNo58Ejw

RunDevelopment commented 2 months ago

Sorry for not going into detail in my previous comment. I didn't just check the code. I also tested SRFormer_S_RealSR.pth and the older models (though not SRFormer_realworld_W22C180). They all worked fine on my end. (Or rather, I didn't see any strange artifacts in the outputs.)

So any artifacts you're getting are likely just the result of your image being very different from the images the models were trained on. This means that either the training dataset wasn't very good or that the architecture generalizes poorly. Neither of those issues can be fixed by spandrel. This is why I said that there's nothing we can do.

zelenooki87 commented 2 months ago

I just tried running the srformer model in conda environment and the model doesn't produce output artifacts for the same input files. Author of srformer model changed something in dictionary of loading ???( see here https://github.com/HVision-NKU/SRFormer/issues/29#issuecomment-2053718012 ) While I won't investigate this further, the model itself doesn't seem particularly superior anyway, so just close issue.