[Bug]: RealESGRAN pure black output

[crusader290]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Allow us to use RealESGRAN models for Hires. fix and Upscaling.

Proposed workflow

1. Go to Extras
2. Select a RealESGRAN Upscaler (.e.g; R-ESRGAN 4x+ Anime6B)
3. Not get a pure black output file consisting of 5kb

Additional information

No response

[lllyasviel]

i cannot reproduce anyone else has this problem?

[crusader290]

Upscale script freed memory successfully.
Downloading: "https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth" to C:\StableDiffusionForge\webui\models\RealESRGAN\RealESRGAN_x4plus_anime_6B.pth

100%|███████████████████████████████████████████████████████████| 17.1M/17.1M [00:01<00:00, 10.5MB/s]
tiled upscale:   0%|                                                          | 0/15 [00:00<?, ?it/s]C:\StableDiffusionForge\webui\modules\upscaler_utils.py:33: RuntimeWarning: invalid value encountered in cast
  arr = arr.round().astype(np.uint8)
tiled upscale: 100%|█████████████████████████████████████████████████| 15/15 [00:11<00:00,  1.30it/s]
Upscale script freed memory successfully.
Downloading: "https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth" to C:\StableDiffusionForge\webui\models\RealESRGAN\RealESRGAN_x4plus.pth

100%|███████████████████████████████████████████████████████████| 63.9M/63.9M [00:04<00:00, 13.7MB/s]
tiled upscale:   0%|                                                          | 0/15 [00:00<?, ?it/s]C:\StableDiffusionForge\webui\modules\upscaler_utils.py:33: RuntimeWarning: invalid value encountered in cast
  arr = arr.round().astype(np.uint8)
tiled upscale: 100%|█████████████████████████████████████████████████| 15/15 [00:36<00:00,  `2.45s/it]

[crusader290]

Every other upscaling model works fine even ESGRAN_4x.

But when I try to use RealESGRAN models I get

C:\StableDiffusionForge\webui\modules\upscaler_utils.py:33: RuntimeWarning: invalid value encountered in cast
arr = arr.round().astype(np.uint8)

[lllyasviel]

the codes for RealESGRAN are exactly same with Automatic1111 maybe you will have same problem with webui dev branch

[crusader290]

the codes for RealESGRAN are exactly same with Automatic1111 maybe you will have same problem with webui dev branch

It's strange because I don't get this problem with Automatic1111

[lllyasviel]

the codes for RealESGRAN are exactly same with Automatic1111 maybe you will have same problem with webui dev branch

It's strange because I don't get this problem with Automatic1111

is your original webui on dev branch or master branch? if webui master branch does not have this problem but webui dev branch has it then it can be reported to webui

[crusader290]

the codes for RealESGRAN are exactly same with Automatic1111 maybe you will have same problem with webui dev branch

It's strange because I don't get this problem with Automatic1111

is your original webui on dev branch or master branch? if webui master branch does not have this problem but webui dev branch has it then it can be reported to webui

I think it's on master branch.

[Zerg17]

I have the same problem with RealESRGAN_x4plus and RealESRGAN_x4plus_anime_6B. The image is completely black. Everything worked on Automatic1111.

[crusader290]

I have the same problem with RealESRGAN_x4plus and RealESRGAN_x4plus_anime_6B. The image is completely black. Everything worked on Automatic1111.

Glad to know it's not just me.

[lllyasviel]

@Zerg17 @crusader290 can someone test with original webui dev branch and see if this is related to forge

[Zerg17]

@lllyasviel I tried https://github.com/AUTOMATIC1111/stable-diffusion-webui , RealESRGAN_x4plus and RealESRGAN_x4plus_anime_6B. Both options work.

[Zerg17]

@lllyasviel a message is displayed at startup

:\programs\webui_forge_cu121_torch21\webui\modules\upscaler_utils.py:33: RuntimeWarning: invalid value encountered in cast
  arr = arr.round().astype(np.uint8)

[onlingr]

I also encountered the same problem The following error content: :\webui_forge_cu121_torch21\webui\modules\upscaler_utils.py:33: RuntimeWarning: invalid value encountered in cast arr = arr.round().astype(np.uint8) [-] ADetailer: img2img inpainting detected. adetailer disabled. ** Error running process: D:\webui_forge_cu121_torch21\webui\extensions\sd-dynamic-prompts\scripts\dynamic_prompting.py Traceback (most recent call last): File "D:\webui_forge_cu121_torch21\webui\modules\scripts.py", line 803, in process script.process(p, script_args) File "D:\webui_forge_cu121_torch21\webui\extensions\sd-dynamic-prompts\sd_dynamic_prompts\dynamic_prompting.py", line 481, in process all_prompts, all_negative_prompts = generate_prompts( File "D:\webui_forge_cu121_torch21\webui\extensions\sd-dynamic-prompts\sd_dynamic_prompts\helpers.py", line 93, in generate_prompts all_prompts = prompt_generator.generate(prompt, num_prompts, seeds=seeds) or [""] File "D:\webui_forge_cu121_torch21\system\python\lib\site-packages\dynamicprompts\generators\randomprompt.py", line 71, in generate prompts.append(str(next(iter(gen)))) StopIteration

[hinablue]

UPDATE:

I found the custom function name I used call _postprocess_image will make the next generator job's sampler fail. And I change the name to another, it work fine. 🧐🧐🧐

BUT, WHY?

---

I've done a lot of testing for this, mostly rewriting the ADetailer extension. A very strange situation, after getting the last image, the two models mediapipe and ultralytics target face detection, and using ultralytics causes an anomaly.

The flow I do the test,

1. Use postprocess_image to get the StableDiffusionProcessing and PostprocessImageArgs.
2. Using ultralytics (YOLO model) to predict the masks.
3. Assign a new img2img processor using StableDiffusionProcessingImg2Img and add the masks.
4. After all, It works at the very first time.
5. And hit the generate again, it will be failed with x_sample = x_sample.astype(np.uint8) in modules/processing.py:973
6. And Forge completely fails and will always produce black image if not restarted.

I follow the source code and I found the second time I hit the generate, in the source of the KDiffusionSampler.sample will return the empty tensor.

AND, when I use the mediapipe model in the step 2, and all work fine.

I have no idea about this.

[Alittfre]

I have the same problem.

[alikecat]

I also have the same problem. Has anyone found the cause of the problem or a solution after the past three months? I'm going to try to reproduce this problem on my other devices.

[alikecat]

On another device, I didn't find this problem. I guess this problem may be related to the graphics card model of the device. The device I used before used a GTX1660s graphics card. In Automatic1111, I need to append the --no-half parameter, while this is not necessary in the forge version. Could this possibly have caused this problem? Since I cannot currently use the device where I previously discovered this problem to verify this guess, I can only hope to give some inspiration to the experts who are trying to solve this problem.

[crusader290]

On another device, I didn't find this problem. I guess this problem may be related to the graphics card model of the device. The device I used before used a GTX1660s graphics card. In Automatic1111, I need to append the --no-half parameter, while this is not necessary in the forge version. Could this possibly have caused this problem? Since I cannot currently use the device where I previously discovered this problem to verify this guess, I can only hope to give some inspiration to the experts who are trying to solve this problem.

I think you're right. I used the same parameters on automatic1111's. And am using a GTX 1660 Super. Will add the parameter and test it out.

[crusader290]

Added the "--no-half --no-half-vae" arguments and can confirm it is now working!