[Bug]: ESRGAN output looks pixelated

allo- commented 2 years ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits

What happened?

When upscaling with ESRGAN, the images looks more pixalated than they should. I have another installation of the plain RealESRGAN project and it provides smooth outlines for anime-style images, where the ESRGAN integrated in the webui generates noisy lines.

Steps to reproduce the problem

Create a image with clear outlines
Go to the extras tab and upscale it
ESRGAN (and some others, but I do not know reference results for them) have outputs that look pixelated at edges.

What should have happened?

The results should be comparable to the upstream project, I think.

Commit where the problem happens

737eb28

What platforms do you use to access UI ?

Linux

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

No response

Additional information, context and logs

I tried BSRGAN in the past and got much better results for anime-style anyway, but either it got removed or is not shown anymore for some other reason. I use https://github.com/xinntao/Real-ESRGAN for comparision to see that it produces smoother outlines, when run without any parameters.

victorca25 commented 2 years ago

Just place the BSRGAN model in the ESRGAN model directory and it will work the same as before.

Can you add images with examples of the issue you have?

allo- commented 2 years ago

original webui_esrgan real_esrgan

Have a look at the skin or the eyes. The esrgan from the webui introduces noise while real-esrgan looks smooth.

Centurion-Rome commented 2 years ago

Maybe include BSRGAN als default in this repo also as it was in the past ?

victorca25 commented 2 years ago

@allo- are you using fp16?

allo- commented 2 years ago

I think everything related to this is on default settings. Especially I did not configure fp16 anywhere.

judgeou commented 2 years ago

I download https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-general-x4v3.pth and put in ESRGAN folder，it looks better than default, and more faster

Splendide-Imaginarius commented 1 year ago

I think I may be running into the same bug. Sample images below, using the RealESRGAN_x4plus model:

Original image: https://github.com/Splendide-Imaginarius/stable-diffusion-webui/blob/bug-report-esrgan-corruption/samples/original.png

Upscaled with official Real-ESRGAN implementation: https://github.com/Splendide-Imaginarius/stable-diffusion-webui/blob/bug-report-esrgan-corruption/samples/real-esrgan-4x-official.png

Upscaled with WebUI Extras tab: https://github.com/Splendide-Imaginarius/stable-diffusion-webui/blob/bug-report-esrgan-corruption/samples/real-esrgan-4x-webui.png

Tiling was disabled in both implementations; the official Real-ESRGAN implementation was run with --fp32; the WebUI was run with --no-half. In WebUI, Upscaler 2 was set to None, and all slider bars for Upscaler 2 and face restoration were set to 0. Running WebUI commit 22bcc7be428c94e9408f589966c2040187245d81 on Windows+Firefox; both implementations are using CPU inference.

Observable differences between the images:

The general color of the foliage near the bottom of the image is mildly different.
The foliage at the bottom of the image produced by the official Real-ESRGAN implementation has much finer detail, while the WebUI implementation produces blurrier foliage.
The white X pattern at upscaled coordinates 1391,1053 is fairly clean in the official Real-ESRGAN implementation, while the WebUI produces a much noisier pattern.
The diagonal white line from upscaled coordinates 360,1106 to 508,1064 is easily visible in the official Real-ESRGAN implementation, while the WebUI mostly blurs it out.

Happy to provide more details or run additional tests on request. If you think this is a different bug, let me know and I'll file a separate issue.

AUTOMATIC1111 / stable-diffusion-webui