[Feature Request]: sberbank-ai/ai-forever's RealESRGAN imp

[Sazoji]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Ru-Dalle Trained it's own RealESRGAN implementation last year specifically for faces and synthetic data, performs similarly to BSRGAN (potentially due to the augments which dive it's aggressive debluring). The implementation of their RealESRGAN is different from the official xintao release, since it supports 2x and 8x, the latter with a particularly interesting model. Combining this option into the hd fix could be useful since you can then downsample to the correct res, or choose a more appropriate scale factor. This model works best with more abstract stuff, but is generally a good option if the implementation of RealESRGAN can load the weights.

Proposed workflow

1. Same position with the other RealESRGAN imps as something like rudalle-sr or sberbank-Resrgan, or load with the custom models
2. Selectable as an upscaler in HD Fix
3. potentally support some math to choose 2x, 4x, and 8x versions if the modified arch is implemented

Additional information

Just asking as a member of the upscale wiki discord, I probably could run this myself in chaiNNer, then use I2I, but the model was designed around upscaling this sort of stuff. This isn't the same issue as merge 2067 as the arch is changed to run those russian real-esrgan models

[victorca25]

Hello again @Sazoji! Can you try putting the model in the ESRGAN directory (not real-ESRGAN) in the current version of the repo?

A PR with my code for the network was merged, so now all ESRGAN models from the DB (and real-ESRGAN) should work fine. Give it a go and let me know if it doesn't work, I can try taking a look if there are other changes in the architecture 😄

[Sazoji]

Hello again @Sazoji! Can you try putting the model in the ESRGAN directory (not real-ESRGAN) in the current version of the repo?

A PR with my code for the network was merged, so now all ESRGAN models from the DB (and real-ESRGAN) should work fine. Give it a go and let me know if it doesn't work, I can try taking a look if there are other changes in the architecture smile

It loads when placed in the /models/ESRGAN folder, but you can't set the scale to 8x nor does it load properly, this model is the weird one that uses pixel-unshuffle to do the upscaling.

8x model, renamed to sberbankESRGAN: https://huggingface.co/sberbank-ai/Real-ESRGAN/blob/main/RealESRGAN_x8.pth

the 2x, 4x performs properly, as well as 8x ESRGAN models in my pull

[victorca25]

Took a look at the architecture, they just added an additional up_conv at the end of the network, making it 4 x 2 = 8, so no major changes. Would be trivial to add to the existing network, I can add it if I submit an additional PR with other requests.

Regarding the maximum scale, I mentioned it in the PR, that's part of the UI code AFAIK, but I don't think it's too difficult to change, so might as well try to change it as well.

[victorca25]

@Sazoji today I got to test it and it works with minimal changes:

I'll submit it in a PR later if I can include other requests.

[victorca25]

@Sazoji please test https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/3976 and give any feedback on how it goes

[Sazoji]

the model arch works, however the UI hasn't updated to show 8x since checking out 3976, even when reloading ui.py BSRGAN also moves to the correct folder without issues

[Sazoji]

yea it's ui-config.json a clean install wouldn't have this issue then well this issue is cleared with that PR, thanks Vic!

[kaneda2004]

Any progress on implementing this 8x model? It works in chaiNNer so far - interested to see it working here - thanks for your contributions

[Sazoji]

Any progress on implementing this 8x model? It works in chaiNNer so far - interested to see it working here - thanks for your contributions

the patch was merged recently, so if you're not seeing 8x, delete ui-config.json and git pull for the UI to support it