[Bug]: RuntimeWarning: invalid value encountered in cast

[ComplexRobot]

Describe the bug

Sometimes after using ADetailer or u DDetailer, the next time I generate an image, this error happens:

G:\stable-diffusion-webui\modules\sd_samplers_common.py:75: RuntimeWarning: invalid value encountered in cast
  x_sample = x_sample.astype(np.uint8)
G:\stable-diffusion-webui\modules\processing.py:1011: RuntimeWarning: invalid value encountered in cast
  x_sample = x_sample.astype(np.uint8)

The result image will then be completely black and from then on all generated images will be blank black images (with or without detailer enabled). I am using the newest version of stability ui forge. This bug has seemed to occur in forge for a long time and only happens after I use ADetailer or u DDetailer.

This error occurs as long as the ADetailer extension is installed. If I disable the ADetailer extension (unchecked in the Extensions tab + Apply and restart UI) the bug goes away. Also, I've noticed the generated image output before inpainting is slightly different with ADetailer installed. (All the same generation parameters, but having ADetailer installed changes how it's generated.)

(Sorry, I do not have the full console log, because reproducing the bug is semi-random.)

Steps to reproduce

• Install ADetailer
• Use a detailer to inpaint an image
• Generate another image, with or without detailer enabled
• "invalid value encountered in cast": Image will be completely black, as well as any image after

Screenshots

No response

Console logs, from start to end.

venv "G:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: f2.0.1v1.10.1-previous-571-g1ef54368
Commit hash: 1ef5436833ae5438ff4bba5cf586f041cc711e21
...
G:\stable-diffusion-webui\modules\sd_samplers_common.py:75: RuntimeWarning: invalid value encountered in cast
  x_sample = x_sample.astype(np.uint8)
G:\stable-diffusion-webui\modules\processing.py:1011: RuntimeWarning: invalid value encountered in cast
  x_sample = x_sample.astype(np.uint8)
...

List of installed extensions

Stable-Diffusion-Webui-Civitai-Helper sd-dynamic-prompts sd-webui-infinite-image-browsing stable-diffusion-webui-state uddetailer

[dosubot[bot]]

Hey there, @ComplexRobot! I'm Dosu, a bot here to help you with bug fixes, questions, and becoming a contributor. Let's take a look at the issue you've reported.

The "invalid value encountered in cast" warning you're experiencing is likely due to the presence of NaN (Not a Number) or infinite values in the image data when attempting to cast it to np.uint8. This can result in the generation of completely black images.

ADetailer modifies image data through denoising and inpainting processes, which can potentially lead to invalid values if not handled correctly ^[1]. The astype(np.uint8) operation does not handle NaN or infinite values gracefully, which can cause this warning.

To address this issue, you can try the following:

1. Check Input Data: Ensure that the input data to the astype(np.uint8) operation does not contain NaN or infinite values. You might need to add checks or preprocessing steps to handle these values before casting.

2. Error Handling: Implement error handling around the casting operation to catch and manage any invalid values. This could involve setting NaN or infinite values to a default value before casting.

3. Review Parameters: Adjust the parameters used in ADetailer's inpainting process to see if different settings prevent the occurrence of invalid values. For example, tweaking denoising_strength or other related parameters might help ^[2].

These steps should help mitigate the issue and prevent the generation of black images.

To continue talking to Dosu, mention @dosu.

[PetteriPaju]

I am also using Forge (f2.0.1v1.10.1-previous-591-gb592142f) Using Adetailer version: v24.9.0 XL-based model.

I have a same issue as OP, it has been issue for me quite a while and it always seemed really random. I have knows for while that at least in my case the error is caused by Prompt Scheduling , but it didn't always trigger.

Now I finally found out a fairly consistent way to trigger the bug: For me it is caused by using more that one Prompt Schedules at same time. (May also be called knows as prompt editing IDK...)

So to add to OPs steps:

Steps to reproduce

1. Install ADetailer
2. Launch/relaunch backend.
3. Use positive prompt with at least 2 seperate prompt-scheduling operations.
4. Generate image with Adetailer Enabled.
5. Pre-Adetailer image generates correctly (including Hires-fix, btw).
6. "invalid value encountered in cast": Adetailer image will be completely black or (In rare cases generation may succeed), as well as any image after, even if Adetailer is disabled or prompt altered.
7. Relaunching Forge backend fixes the issue, reloading the web page is not always needed, but recommended.

This prompt works fine:

megami magazine, score_9, [anime:3d:0.3],

But this prompt triggers the bug:

megami magazine, score_9, [anime:3d:0.3], [1girl:1boy:0.2],

Now here is another oddity

1. Install Adetailer
2. Launch/relaunch backend.
3. Generate image with Adetailer enabled, but only with 1 prompt schedules (or none at all)
4. Image generates successfully
5. Optionally toggle off Adetailer (seems to make no difference at this point)
6. Alter prompt to have at least two prompt schedules and press Generate
7. All generated images are black even before reaching Adetailer.

So even perfectly successful Adetailer operation without prompt scheduling can create conditions for for the bug to trigger, even if Adetailer is no longer enabled. Weird.....

Key seems to be that Adetailer must run at least once.

Other observations:

• I tested this a lot and sometimes for no reason the generation just seems to work and I can't understand why. Adding more prompt schedules or just altering prompt triggers the bug sooner or later.
• I couldn't get the bug to trigger without Adetailer. I tried manual Inpainting and Img2Img.
• Having prompt schedules in Negative prompt seems to work fine.
• Sometimes instead of completely black generation you get black image with white box at corner.
• In past testing if batch size >1, I have seen 1st image of batch to generate fine in quite often, but I couldn't reproduce that effect at this time.
• Bug seems to happen even if you manually give Adetailer (in Adetailer menu, 1st, 2nd etc.) a prompt without prompt schedules, thought I don't have time to double check that one right now.
• I tried tinkering with any remotely relevant setting, without luck. Such as:
  • Compatibility/Use old prompt editing timelines.
  • All VAE Settings

[ComplexRobot]

I had been using prompt editing in all my prompts for a while. So this tracks with what I was experiencing.

What I experienced was after generating and running ADetailer one time, the next image I generate will fail with the error with or without ADetailer enabled.

Also, your explanations/testing seems to indicate that ADetailer is hooked into the prompt scheduling code, which may be the key to fixing the bug. Thank you!

EDIT: I was looking into fixing the bug, but it's tough to figure out what's causing it and why. It doesn't seem to make sense that ADetailer would affect generations after it's been used. It doesn't help that the bug is somewhat random and doesn't always happen.