[Bug]: Inpainting suddenly stopped working

mr-september commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits

What happened?

Been using this extensively for a week... today suddenly inpainting stopped working halfway through. All output images are either exactly the input image, or with the blurred mask.

Steps to reproduce the problem

Try to do any inpainting
Get back original image or the with the blurred mask.

What should have happened?

Inpainted image

Commit where the problem happens

fd59537

What platforms do you use to access the UI ?

Windows 11, AMD 5700

What browsers do you use to access the UI ?

Opera

Command Line Arguments

--opt-sub-quad-attention --medvram --disable-nan-check --autolaunch --api --cors-allow-origins=http://127.0.0.1:3456

I have tried enabling these either individually or in combination, to no avail:
--precision full --no-half --opt-split-attention-v1

List of extensions

Console logs

Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Commit hash: fd59537df9420bb14c1d6330ec59e30ce870a481
Installing requirements for Web UI
Installing requirements for Batch Face Swap

Installing requirements 1 for Infinite-Zoom

Interrogations are fallen back to cpu. This doesn't affect on image generation. But if you want to use interrogate (CLIP or DeepBooru), check out this issue: https://github.com/lshqqytiger/stable-diffusion-webui-directml/issues/10
Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled

Installing sd-dynamic-prompts requirements.txt

Installing sd-webui-controlnet requirement: opencv-contrib-python==4.7.0.68

Launching Web UI with arguments: --opt-sub-quad-attention --medvram --opt-split-attention-v1 --disable-nan-check --autolaunch --api --cors-allow-origins=http://127.0.0.1:3456
Interrogations are fallen back to cpu. This doesn't affect on image generation. But if you want to use interrogate (CLIP or DeepBooru), check out this issue: https://github.com/lshqqytiger/stable-diffusion-webui-directml/issues/10
Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled
No module 'xformers'. Proceeding without it.
Loading weights [6482f11700] from E:\stable-diffusion-webui-directml\models\Stable-diffusion\realisticVisionV20_v20-inpainting.safetensors
Creating model from config: E:\stable-diffusion-webui-directml\configs\v1-inpainting-inference.yaml
LatentInpaintDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.54 M params.
Loading VAE weights specified in settings: E:\stable-diffusion-webui-directml\models\VAE\vae-ft-mse-840000-ema-pruned.ckpt
Applying sub-quadratic cross attention optimization.
Textual inversion embeddings loaded(2): easynegative, ng_deepnegative_v1_75t
Model loaded in 2.5s (load weights from disk: 0.2s, create model: 0.5s, apply weights to model: 0.8s, apply half(): 0.7s, load VAE: 0.3s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 16.3s (import torch: 2.2s, import gradio: 1.2s, import ldm: 0.6s, other imports: 1.3s, setup codeformer: 0.2s, load scripts: 2.5s, load SD checkpoint: 2.8s, create ui: 5.2s, gradio launch: 0.2s).
100%|██████████████████████████████████████████████████████████████████████████████████| 13/13 [00:14<00:00,  1.13s/it]
100%|██████████████████████████████████████████████████████████████████████████████████| 13/13 [00:12<00:00,  1.07it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 13/13 [00:16<00:00,  1.23s/it]
100%|██████████████████████████████████████████████████████████████████████████████████| 13/13 [00:13<00:00,  1.00s/it]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 52/52 [01:04<00:00,  1.23s/it]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 52/52 [01:04<00:00,  1.03it/s]

Additional information

I have tried:

Restart webui bat
Restart computer
Fresh git pull
Various commandline_args (detailed above)
Changing checkpoint/models
Adjust literally every single setting under "inpainting"
Deleting the venv folder to force-update everything as suggested in issue 77. torch-directml "Version: 0.1.13.1.dev230413"

revons1984 commented 1 year ago

I found this issue on the main branch as well, the problem is with the VAE, add --no-half-vae to your command line arguments @mr-september

thonsm06 commented 1 year ago

These are my current command line arguments, --autolaunch --opt-sub-quad-attention --opt-split-attention-v1 --no-half-vae --precision full --no-half --disable-nan-check

So far everything is working.

I needed to keep --no-half for inpainting to work.

Windows 10, 6900xt.

Convexity-ai commented 1 year ago

Is the workaround above the only solution to getting inpainting working and are there any plans to investigate this further? This seems specific to using the inpainting feature with half precision. @lshqqytiger

In testing the following behaviour occurs:

Fill: The targeted area remains pale and no filling is done. Original: Slight burn is seen on the targeted area but no variance in the image is made. Latent noise: The targeted area remains pale and no variance in the image is made. Latent nothing: The targeted area remains pale and no variance in the image is made.

This issue doesn't seem to be model, VAE, or sampler specific either.

Build: 108ada8 Arguments used: --medvram --opt-sub-quad-attention

Ideally, I'd like to use half precision for image generation / inpainting due to speed/VRAM use, but so far the only way I can get inpainting to work at all is to add the --no-half argument.

lshqqytiger commented 1 year ago

Because DirectML does not natively support autocast, I implemented it by myself. So, it can be quite different from its native implementation of torch and hardly provide any improvement on performance or VRAM side. I can't access the source code of DirectML because it is not an open-source software, I can't do much about half precision.

Bod9001 commented 1 year ago

when you run with without --no-half, in painting is very strange, it looks like what it does though is

any pixels along the top of the image stretch down all the way, then the mask is rotated by 90° grid-0019

Hopefully this can narrow down the issue

mr-september commented 1 year ago

@Bod9001 that's a very interesting find!

Some quick updates on my end: Maybe it's part of the teething issues from Auto1111/DirectML, but this issue (inpainting not working) and other issues (img-2-img not working, sometimes the "show extra networks" doesn't work) seem to randomly come and go. No updates or restarts or whatevers apparently fix or cause them. Truly very mysterious.

Yeetmeister92 commented 1 year ago

I found this issue on the main branch as well, the problem is with the VAE, add --no-half-vae to your command line arguments @mr-september

Hey Im a big noob. How do I just add stuff to the command line arguments?

Im on windows 10 and 6700xt GPU. Gives me the same blur

Elise96nl commented 1 month ago

redownload models. put in in seperate folder (fixed models) and give it a hard read only attribute. Use seperate checkpoint for adetailer, so it has to reload between txt2image and adetailer. also allow only 1 model in memory.

I've had no nans errors after doing that

lshqqytiger / stable-diffusion-webui-amdgpu