Closed glonlas closed 1 year ago
I confirm the issue is visible in the current development
branch. It happens because either it cannot load the Real-ESRGAN python Library (MPS) or cannot load a model for x3 upscaling on CUDA..
This PR fix it for MacOS MPS because it loads Real-ESRGAN from the binary rather the Python Library. When I switch to this PR upscale work.
When on current development
the terminal output is: (Failed)
>> Real-ESRGAN Upscaling seed:1586105133 : scale:2x
>> Error running RealESRGAN or GFPGAN. Your image was not upscaled.
'NoneType' object has no attribute 'enhance'
When I switched to the PR #424 the terminal output is: (Successful)
>> Real-ESRGAN Upscaling seed:1857161085 : scale:2x
127.0.0.1 - - [10/Sep/2022 23:18:09] "GET /outputs/img-samples/000008.1857161085.png HTTP/1.1" 200 -
[0 Apple M1] queueC=0[1] queueG=0[1] queueT=0[1]
[0 Apple M1] bugsbn1=0 bugbilz=97 bugcopc=0 bugihfa=0
[0 Apple M1] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1
[0 Apple M1] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1
0.00%
11.11%
22.22%
33.33%
44.44%
55.56%
66.67%
77.78%
88.89%
>> Usage stats:
>> 1 image(s) generated in 30.51s
>> Max VRAM used for this generation: 0.00G
The issue happens because the code base will load a model for 2x and x4 scaling only.
The error happens in:
>> Real-ESRGAN Upscaling seed:3350596856 : scale:2x
Traceback (most recent call last):
File "/ldm/generate.py", line 467, in upscale_and_reconstruct
image = real_esrgan_upscale(
File "ldm/gfpgan/gfpgan_tools.py", line 149, in real_esrgan_upscale
output, img_mode = upsampler.enhance(
AttributeError: 'NoneType' object has no attribute 'enhance'
>> Error running RealESRGAN or GFPGAN. Your image was not upscaled.
'NoneType' object has no attribute 'enhance'
When we dive deep this happens because of the following code which is executed only on CUDA platform:
model_path = {
2: 'https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth',
4: 'https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth',
}
if upsampler_scale not in model_path:
return None
As we can see it load model for upscale at x2 and x4, but no model for x3 which returns None. This is what generates the 'NoneType' object has no attribute 'enhance'
As of today the x3 cannot work with the implementation of the python lib of Real-ESRGAN. While it works if we use the Bin. @blessedcoolant you were 100% about the non support of x3 upscaling.
We need to revert this PR: https://github.com/lstein/stable-diffusion/pull/442 (I will do the revert)
cc: @lstein
@glonlas could you confirm that upscaling is now working on the Macintosh platform? Please check either "development" HEAD, or the tag release-candidate-1.14.2.
Thnx
For the time being it works (1.14.3). In the future, it may be worth looking into using the executables (there's versions for Linux/Windows/Mac) if these reports I found running it are not isolated to my machine. https://github.com/lstein/stable-diffusion/discussions/511#discussioncomment-3624882 (about quality and time). But anyway, it works
Describe your environment
git status
says anything other than "On branch main" paste it here] developmentgit show
and paste the line that starts with "Merge" here] bfb278127923fbd461c4549a4b7f2f2c1dd34b8cDescribe the bug Error
'NoneType' object has no attribute 'enhance'
when using GFPGAN via the Web UI.To Reproduce On MPS: Steps to reproduce the behavior:
python scripts/dream.py --full_precision --web
Upscaling Level = x2|x3|x4
(GPFGAN Strength
can be anything)On CUDA Steps to reproduce the behavior:
python scripts/dream.py --full_precision --web
Upscaling Level = x3
(GPFGAN Strength
can be anything)The terminal will return
Expected behavior The generation of the image works with GPFGAN and Upscaling without any trouble.