[Bug]: Images are messed up in the last generation step(s) (Euler a, Euler, LMS etc.)

bosbrand commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits

What happened?

With any of my models, generated images are screwed up in the last step(s). I can see the generation doing great when I run a script, outputting every step until the last steps. Then it is as if there was a sort of sharpening taking place in certain places, most noticeably faces. I see sharpening, but it is more like distorting. In LMS this effect is most apparent, because there the problem areas are just made into glitchy mosaic in the last steps.

Steps to reproduce the problem

Start stable diffusion
Choose Model
Input prompts, set size, choose steps (doesn't matter how many, but maybe with fewer steps the problem is worse), cfg scale doesn't matter too much (within limits)
Run the generation
look at the output with step by step preview on.

What should have happened?

The last step should improve on the ones before, except now it tends to ruin what was building up beautifully.

Commit where the problem happens

645f4e7ef8c9d59deea7091a22373b2da2b780f2

What platforms do you use to access UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--xformers

Additional information, context and logs

sample-00035 sample-00033

Greendayle commented 1 year ago

Maybe it was always like this??

On Thu, Apr 20, 2023, 14:45 Jan Wilmans @.***> wrote:

Tested with different commandline options:

set COMMANDLINE_ARGS=--no-half --no-half-vae --xformers --autolaunch set COMMANDLINE_ARGS=--no-half-vae --xformers set COMMANDLINE_ARGS=--autolaunch set COMMANDLINE_ARGS=

no change. Still the same distorted output

— Reply to this email directly, view it on GitHub https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/7244#issuecomment-1516264180, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATQVTSYL4OLNWVFATR5XPE3XCEVU5ANCNFSM6AAAAAAUHQKYXA . You are receiving this because you commented.Message ID: @.***>

janwilmans commented 1 year ago

Yes, I'm beginning to think that too, but it still looks like a bug too me, what do you think?

Greendayle commented 1 year ago

I also did some experiments some time ago and could generate identical pictures with new and old versions, when using correct .vae.pt, and making sure it loaded. But still I do have vague memory that "it was better long time ago".

These corruptions are worrysome still.

On Thu, Apr 20, 2023, 14:58 Jan Wilmans @.***> wrote:

Yes, I'm beginning to think that too, but it still looks like a bug too me, what do you think?

— Reply to this email directly, view it on GitHub https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/7244#issuecomment-1516283522, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATQVTS4TURJAFD4QV2DQKYTXCEXIZANCNFSM6AAAAAAUHQKYXA . You are receiving this because you commented.Message ID: @.***>

jconorgrogan commented 1 year ago

Having the same issue- Ive found that this does not occur when I input low CFG (eg 1.5, 2). It also doesnt happen with hirez-fix is enabled (though obviously its a new image at that point)

janwilmans commented 1 year ago

Collecting torch==1.13.1+cu117
  Using cached https://download.pytorch.org/whl/cu117/torch-1.13.1%2Bcu117-cp310-cp310-win_amd64.whl (2255.4 MB)
Collecting torchvision==0.14.1+cu117

tested with older transformers version 4.20.1 (June 2022), with torch==1.13.1+cu117 and torchvision==0.14.1+cu117

no surprise: exactly the same broken result

janwilmans commented 1 year ago

@jconorgrogan interesting, at CFG 2 it does seem to disappear.

dragon's eye, (close up:1.3), wide anime eyes Negative prompt: bw, bad hands, (blurry:1.2), duplicate, (duplicate body parts:1.2), disfigured, poorly drawn, extra limbs, fused fingers, extra fingers, twisted, malformed hands, low quality Steps: 10, Sampler: LMS, CFG scale: 2, Seed: 1591058834, Size: 512x768, Model hash: fbcf965a62, Model: move_anything-v4.5

janwilmans commented 1 year ago

00089-1591058834

dragon's eye, (close up:1.3), wide anime eyes
Negative prompt: bw, bad hands, (blurry:1.2), duplicate, (duplicate body parts:1.2), disfigured, poorly drawn, extra limbs, fused fingers, extra fingers, twisted, malformed hands, low quality
Steps: 10, Sampler: LMS, CFG scale: 7, Seed: 1591058834, Size: 512x768, Model hash: fbcf965a62, Model: move_anything-v4.5

janwilmans commented 1 year ago

So very odd, I used https://github.com/AlUlkesh/sd_save_intermediate_images to save all the images and create the GIF above, but from 10 steps it saves only 9 ? Not sure if that is a bug in 'sd_save_intermediate_images', is this somehow related to c53852e257a1fd7daa5f6a8415d17645d5ffef87 ?

janwilmans commented 1 year ago

possibly related issues #1817, #3483, #7077

janwilmans commented 1 year ago

Interesting case:

left: N-1 frame, right: final output image

somehow it looks as if some parts of the picture get more detail, but others overshoot the target?

superglue, box, ring, heliotrope, coffee helicopter, boring heart, reflective
Negative prompt: (bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5,(NG_DeepNegative_V1_75T:1.3)
Steps: 10, Sampler: LMS, CFG scale: 7, Seed: 1034228667, Size: 512x512, Model hash: 0466ae65a0, Model: move_epicDiffusion11

JD1234JD1234 commented 1 year ago

@janwilmans I remember that this issue started exactly on the day I first saw 'torch==1.13' maybe try even before that

janwilmans commented 1 year ago

testing with torchvision==0.13.1

first uninstall clean-fid, requests, torch and torchvision
to install an old torchvision version, you need to make sure to install a special version compiled with cuda included:
```
pip3 install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
```
no difference, same corruption

janwilmans commented 1 year ago

Random thought: could it be that UTF-8 characters in the prompt mess up the image? many prompts are copied from websites and then modified. I tried to look carefully are my reproduction prompts, but they look ASCII only to me.

robproctor commented 1 year ago

Random thought: could it be that UTF-8 characters in the prompt mess up the image? many prompts are copied from websites and then modified. I tried to look carefully are my reproduction prompts, but they look ASCII only to me.

Janwilmans, what do you mean they look ASCII? I'm not an encoding expert or anything, but my understanding is that the characters are encoded within python, not from what is typed into the prompt. Are you using some localization extension?

Additionally I was having a similar possible issue which is why I found this thread. Reading through your notes, I have tried to recreate that articular issue you are having and am unable to recreate the exact image. You may want to enable the full console report of the generation parameters including the hypernetwork (settings > system) to see more info and if it matches with mine below.

However, it may be better to just clear all prompt info and simply use a single easy token like cat, with no negatives, hypernets/loras, etc and either grab a fresh copy of the model or the standard sd model. After that it will be easier narrow things down.

00580-1034228667

{"prompt": "superglue, box, ring, heliotrope, coffee helicopter, boring heart, reflective\n", "all_prompts": ["superglue, box, ring, heliotrope, coffee helicopter, boring heart, reflective\n"], "negative_prompt": " (bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5,(NG_DeepNegative_V1_75T:1.3)", "all_negative_prompts": [" (bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5,(NG_DeepNegative_V1_75T:1.3)"], "seed": 1034228667, "all_seeds": [1034228667], "subseed": 1960961699, "all_subseeds": [1960961699], "subseed_strength": 0, "width": 512, "height": 512, "sampler_name": "LMS", "cfg_scale": 7, "steps": 10, "batch_size": 1, "restore_faces": false, "face_restoration_model": null, "sd_model_hash": "0466ae65a0", "seed_resize_from_w": 0, "seed_resize_from_h": 0, "denoising_strength": null, "extra_generation_params": {}, "index_of_first_image": 0, "infotexts": ["superglue, box, ring, heliotrope, coffee helicopter, boring heart, reflective\n\nNegative prompt: (bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5,(NG_DeepNegative_V1_75T:1.3)\nSteps: 10, Sampler: LMS, CFG scale: 7, Seed: 1034228667, Size: 512x512, Model hash: 0466ae65a0, Model: epicDiffusion_epicDiffusion11"], "styles": [], "job_timestamp": "20230420190856", "clip_skip": 1, "is_using_inpainting_conditioning": false}

janwilmans commented 1 year ago

@robproctor could you do a venv\Scripts\pip3 list to see what specific versions of libraries you are using?

robproctor commented 1 year ago

@robproctor could you do a venv\Scripts\pip3 list to see what specific versions of libraries you are using?

Sorry, got sidetracked, fixed my other problem :) Hope this helps.

Package           Version
----------------- -------
mpmath            1.3.0
pip               22.2.1
setuptools        63.2.0
sympy             1.11.1
typing_extensions 4.5.0

janwilmans commented 1 year ago

No, somehow you didn't execute the command in the right virtual environment, the list is 120+ entries and contained torch and torchvision among others

try this please:

janwilmans commented 1 year ago

Fresh attempt:

uninstall everything, conda, python, etc, all python.exe's
install python 3.10.6 from https://www.python.org/downloads/release/python-3106/, leave all options default, just next, next... finish.
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git at hash 22bcc7be428c94e9408f589966c2040187245d81
edit webui-user.bat to specify set PYTHON=C:\Users\Jan\AppData\Local\Programs\Python\Python310\python.exe explicitly
run webui-user.bat (installs cleanly and automatically downloads checkpoint v1-5-pruned-emaonly.safetensors [6ce0161689]

enter prompt:

superman ring
Steps: 10, Sampler: LMS, CFG scale: 7, Seed: 42, Size: 512x512, Model hash: 6ce0161689, Model: v1-5-pruned-emaonly

00004-42

janwilmans commented 1 year ago

You might be temped to blame LMS, I have seen comments like "just don't use LMS". However, I'm convinced that there are either multiple problems play here, or, it is just more visible using LMS.

In the tests I did so far, I have seen the problem on "Eular a" and "DPM++ 2M Karras" also. And just to be clear: "the problem" is that the N-1 image is objectively better compared to the final output.

Now, with objectively better, I do not mean to say that some models might be overtrained or the prompt is somehow over-fitted. I mean the problem is visible only in the final output, and if it was a matter of over-shooting the fit, it would be unlikely to not also show up in N-2, N-3 also, getting progressively worse.

left: N-1, (actually, mostly identical to N-2 and N-3) right: the final image, its more 'grainy, details from the ears and blush are gone and the colors are flushed out.

janwilmans commented 1 year ago

I think so far we can say the following:

for some images, the N-1 step objectively yields an result that is closer to the prompt compared to the output image.
possibly related issues https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1817, https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/3483, https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/7077
with LMS is pretty easy to demonstrate this, there are definitely examples for but Euler, Euler Ancestral and DPM++ 2M Karras as well.
The root cause is unclear to me, but I think from all the experiments it is safe to say:
- it does not seem to be related to the text-input prompt; we have demonstrated that prompts with and with negative prompt produce have this effect, and we made sure that only well formed strings were used without any non-ascii characters. And without empty ,, entries or mismatched braces
- it is not a recently introduced problem; we have demonstrated that all version of automatic 1111 since Aug 2022 (initial commit) had the same behavior, I tested with torch==1.12.1+cu113 torchvision==0.13.1+cu113, also dated Aug 2022. However, this user reported it was introduced in January 2023. This could still be true, but maybe its not 'introduced' in Automatic1111, but introduced in checkpoints, VAE, Cuda, or drivers?

Things to be tested:

I wonder if it could be driver-related, I have modest hardware (RTX 1660 Super, 6 GB VRAM)

If someone wants to help, please try the prompt below (NSFW) and please report the result and the used hardware. You can see here what it's suppose to look like

I will try to locate a SFW example with Euler A

masterpiece,cowboy pretty face
Negative prompt: (bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5,(NG_DeepNegative_V1_75T:1.3)
Steps: 10, Sampler: Euler a, CFG scale: 8, Seed: 901305577, Size: 512x512, Model hash: e714ee20aa, Model: abyssorangemix2_Hard
Template: masterpiece,cowboy pretty face
Negative Template: (bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5,(NG_DeepNegative_V1_75T:1.3)

Jhonthe7th commented 1 year ago

Stumbled across the same issue just recently. Huge thnx. for the research already into this @janwilmans . Something I can add from the testing I did the last hour, it feels like it is heavily linked to the negative-prompt. Also the negative-prompt token counter behaves weirdly. Depending on the arangement of the same tags + TIs, the count changes, which should not be the case in my opinion?

Used the following list of prompts:

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "bad-artist" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "NG_DeepNegative_V1_75T" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "bad-hands-5" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "bad-picture-chill-75v" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "bad-artist, NG_DeepNegative_V1_75T, bad-hands-5, bad-picture-chill-75v" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

Which resulted in the following output:

As visible, the "bad-hands-5" and "bad-picture-chill-75v" triggered the effect especially bad.

Hope someone with more hands on experience and internal understanding can use this.

robproctor commented 1 year ago

No, somehow you didn't execute the command in the right virtual environment, the list is 120+ entries and contained torch and torchvision among others

try this please:

Sorry, here is the full list


---------------------------- ---------------
absl-py                      1.4.0
accelerate                   0.12.0
addict                       2.4.0
aenum                        3.1.11
aiofiles                     23.1.0
aiohttp                      3.8.4
aiosignal                    1.3.1
albumentations               1.3.0
altair                       4.2.2
antlr4-python3-runtime       4.9.3
anyio                        3.6.2
astunparse                   1.6.3
async-timeout                4.0.2
attrs                        22.2.0
basicsr                      1.4.2
beautifulsoup4               4.11.2
bitsandbytes                 0.35.0
blendmodes                   2022
boltons                      23.0.0
cachetools                   5.3.0
certifi                      2022.12.7
chardet                      4.0.0
charset-normalizer           3.0.1
clean-fid                    0.1.29
click                        8.1.3
clip                         1.0
colorama                     0.4.6
contourpy                    1.0.7
cycler                       0.11.0
deprecation                  2.1.0
diffusers                    0.10.2
discord-webhook              1.0.0
einops                       0.4.1
entrypoints                  0.4
facexlib                     0.2.5
fastapi                      0.90.1
ffmpy                        0.3.0
filelock                     3.9.0
filterpy                     1.4.5
flatbuffers                  23.3.3
font-roboto                  0.0.1
fonts                        0.0.3
fonttools                    4.38.0
frozenlist                   1.3.3
fsspec                       2023.1.0
ftfy                         6.1.1
future                       0.18.3
gast                         0.4.0
gdown                        4.6.4
gfpgan                       1.3.8
gitdb                        4.0.10
GitPython                    3.1.27
google-auth                  2.16.1
google-auth-oauthlib         0.4.6
google-pasta                 0.2.0
gradio                       3.16.2
grpcio                       1.51.3
h11                          0.12.0
h5py                         3.8.0
httpcore                     0.15.0
httpx                        0.23.3
huggingface-hub              0.12.1
idna                         2.10
imageio                      2.26.0
importlib-metadata           6.0.0
inflection                   0.5.1
Jinja2                       3.1.2
joblib                       1.2.0
jsonmerge                    1.8.0
jsonschema                   4.17.3
keras                        2.11.0
kiwisolver                   1.4.4
kornia                       0.6.7
lark                         1.1.2
lazy_loader                  0.1
libclang                     15.0.6.1
linkify-it-py                2.0.0
lion-pytorch                 0.0.7
llvmlite                     0.39.1
lmdb                         1.4.0
lpips                        0.1.4
Markdown                     3.4.1
markdown-it-py               2.2.0
MarkupSafe                   2.1.2
matplotlib                   3.7.0
mdit-py-plugins              0.3.4
mdurl                        0.1.2
mediapipe                    0.9.1.0
modelcards                   0.1.6
multidict                    6.0.4
mypy-extensions              1.0.0
networkx                     3.0
numba                        0.56.4
numpy                        1.23.3
oauthlib                     3.2.2
omegaconf                    2.2.3
open-clip-torch              2.7.0
opencv-contrib-python        4.7.0.72
opencv-python                4.7.0.72
opencv-python-headless       4.7.0.72
opt-einsum                   3.3.0
orjson                       3.8.7
packaging                    23.0
pandas                       1.5.3
piexif                       1.1.3
Pillow                       9.4.0
pip                          22.2.1
protobuf                     3.19.6
psutil                       5.9.4
pyasn1                       0.4.8
pyasn1-modules               0.2.8
pycryptodome                 3.17
pydantic                     1.10.5
pyDeprecate                  0.3.2
pydub                        0.25.1
pyparsing                    3.0.9
pyre-extensions              0.0.23
pyrsistent                   0.19.3
PySocks                      1.7.1
python-dateutil              2.8.2
python-multipart             0.0.6
pytorch-lightning            1.7.6
pytz                         2022.7.1
PyWavelets                   1.4.1
PyYAML                       6.0
qudida                       0.0.4
realesrgan                   0.3.0
regex                        2022.10.31
requests                     2.25.1
requests-oauthlib            1.3.1
resize-right                 0.0.2
rfc3986                      1.5.0
rsa                          4.9
safetensors                  0.2.7
scikit-image                 0.19.2
scikit-learn                 1.2.1
scipy                        1.10.1
sentencepiece                0.1.97
setuptools                   63.2.0
six                          1.16.0
smmap                        5.0.0
sniffio                      1.3.0
soupsieve                    2.4
starlette                    0.23.1
tb-nightly                   2.12.0a20230126
tensorboard                  2.11.0
tensorboard-data-server      0.6.1
tensorboard-plugin-wit       1.8.1
tensorflow                   2.11.0
tensorflow-estimator         2.11.0
tensorflow-intel             2.11.0
tensorflow-io-gcs-filesystem 0.31.0
termcolor                    2.2.0
threadpoolctl                3.1.0
tifffile                     2023.2.28
timm                         0.6.7
tokenizers                   0.13.2
toolz                        0.12.0
torch                        1.13.1+cu117
torchdiffeq                  0.2.3
torchmetrics                 0.11.3
torchsde                     0.2.5
torchvision                  0.14.1+cu117
tqdm                         4.64.1
trampoline                   0.1.2
transformers                 4.25.1
typing_extensions            4.5.0
typing-inspect               0.8.0
uc-micro-py                  1.0.1
urllib3                      1.26.14
uvicorn                      0.20.0
wcwidth                      0.2.6
websockets                   10.4
Werkzeug                     2.2.3
wheel                        0.38.4
wrapt                        1.15.0
xformers                     0.0.17.dev464
yapf                         0.32.0
yarl                         1.8.2
zipp                         3.15.0```

robproctor commented 1 year ago

Stumbled across the same issue just recently. Huge thnx. for the research already into this @janwilmans . Something I can add from the testing I did the last hour, it feels like it is heavily linked to the negative-prompt. Also the negative-prompt token counter behaves weirdly. Depending on the arangement of the same tags + TIs, the count changes, which should not be the case in my opinion?

Used the following list of prompts:

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "bad-artist" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "NG_DeepNegative_V1_75T" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "bad-hands-5" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "bad-picture-chill-75v" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

--prompt "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality" --negative_prompt "bad-artist, NG_DeepNegative_V1_75T, bad-hands-5, bad-picture-chill-75v" --steps 20 --cfg_scale 8 --sampler_name "Euler a" --seed 3522336841 --width 640 --height 640

Which resulted in the following output:

As visible, the "bad-hands-5" and "bad-picture-chill-75v" triggered the effect especially bad.

Hope someone with more hands on experience and internal understanding can use this.

Hi I have tried to recreate your issue but can not. What model are you using?

Below I have rendered your prompt with bad-picture-chill-75v using the model it was trained on, chilloutmix_NiPrunedFp32Fix.safetensors when I run that model with your prompt I get a different response.

00323-3522336841

Below is the full prompt info:

{"prompt": "mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality", "all_prompts": ["mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality"], "negative_prompt": "bad-picture-chill-75v", "all_negative_prompts": ["bad-picture-chill-75v"], "seed": 3522336841, "all_seeds": [3522336841], "subseed": 4019433547, "all_subseeds": [4019433547], "subseed_strength": 0, "width": 640, "height": 640, "sampler_name": "Euler a", "cfg_scale": 8, "steps": 20, "batch_size": 1, "restore_faces": false, "face_restoration_model": null, "sd_model_hash": "fc2511737a", "seed_resize_from_w": 0, "seed_resize_from_h": 0, "denoising_strength": null, "extra_generation_params": {}, "index_of_first_image": 0, "infotexts": ["mid shot, 1girl, smiling, flower field, cloudy_sky, rim lit, dramatic lighting, incredible quality\nNegative prompt: bad-picture-chill-75v\nSteps: 20, Sampler: Euler a, CFG scale: 8, Seed: 3522336841, Size: 640x640, Model hash: fc2511737a, Model: chilloutmix_NiPrunedFp32Fix"], "styles": [], "job_timestamp": "20230426160212", "clip_skip": 1, "is_using_inpainting_conditioning": false}

janwilmans commented 1 year ago

@robproctor thanks for the pip list, here is mine, is it quite different:

absl-py                 1.4.0
accelerate              0.12.0
addict                  2.4.0
aenum                   3.1.12
aiofiles                23.1.0
aiohttp                 3.8.4
aiosignal               1.3.1
altair                  4.2.2
antlr4-python3-runtime  4.9.3
anyio                   3.6.2
async-timeout           4.0.2
attrs                   23.1.0
basicsr                 1.4.2
beautifulsoup4          4.12.2
blendmodes              2022
blis                    0.7.9
boltons                 23.0.0
cachetools              5.3.0
catalogue               2.0.8
certifi                 2022.12.7
chardet                 4.0.0
charset-normalizer      3.1.0
clean-fid               0.1.29
click                   8.1.3
clip                    1.0
colorama                0.4.6
confection              0.0.4
contourpy               1.0.7
cssselect2              0.7.0
cycler                  0.11.0
cymem                   2.0.7
deprecation             2.1.0
dynamicprompts          0.21.2
einops                  0.4.1
entrypoints             0.4
facexlib                0.3.0
fastapi                 0.94.0
ffmpy                   0.3.0
filelock                3.12.0
filterpy                1.4.5
font-roboto             0.0.1
fonts                   0.0.3
fonttools               4.39.3
frozenlist              1.3.3
fsspec                  2023.4.0
ftfy                    6.1.1
future                  0.18.3
gdown                   4.7.1
gfpgan                  1.3.8
gitdb                   4.0.10
GitPython               3.1.30
google-auth             2.17.3
google-auth-oauthlib    1.0.0
gradio                  3.23.0
grpcio                  1.54.0
h11                     0.12.0
httpcore                0.15.0
httpx                   0.24.0
huggingface-hub         0.13.4
idna                    2.10
imageio                 2.27.0
inflection              0.5.1
Jinja2                  3.1.2
jsonmerge               1.8.0
jsonschema              4.17.3
kiwisolver              1.4.4
kornia                  0.6.7
langcodes               3.3.0
lark                    1.1.2
lazy_loader             0.2
lightning-utilities     0.8.0
linkify-it-py           2.0.0
llvmlite                0.39.1
lmdb                    1.4.1
lpips                   0.1.4
lxml                    4.9.2
Markdown                3.4.3
markdown-it-py          2.2.0
MarkupSafe              2.1.2
matplotlib              3.7.1
mdit-py-plugins         0.3.3
mdurl                   0.1.2
multidict               6.0.4
murmurhash              1.0.9
networkx                3.1
numba                   0.56.4
numpy                   1.23.3
oauthlib                3.2.2
omegaconf               2.2.3
open-clip-torch         2.7.0
opencv-python           4.7.0.72
orjson                  3.8.10
packaging               23.1
pandas                  2.0.0
pathy                   0.10.1
piexif                  1.1.3
Pillow                  9.4.0
pip                     22.2.1
preshed                 3.0.8
protobuf                3.20.0
psutil                  5.9.5
pyasn1                  0.5.0
pyasn1-modules          0.3.0
pydantic                1.10.7
pydub                   0.25.1
pyparsing               3.0.9
pyrsistent              0.19.3
PySocks                 1.7.1
python-dateutil         2.8.2
python-multipart        0.0.6
pytorch-lightning       1.9.4
pytz                    2023.3
PyWavelets              1.4.1
PyYAML                  6.0
realesrgan              0.3.0
regex                   2023.3.23
reportlab               3.6.12
requests                2.28.2
requests-oauthlib       1.3.1
resize-right            0.0.2
rsa                     4.9
safetensors             0.3.0
scikit-image            0.19.2
scipy                   1.10.1
semantic-version        2.10.0
Send2Trash              1.8.0
sentencepiece           0.1.98
setuptools              63.2.0
six                     1.16.0
smart-open              6.3.0
smmap                   5.0.0
sniffio                 1.3.0
soupsieve               2.4.1
spacy                   3.5.2
spacy-legacy            3.0.12
spacy-loggers           1.0.4
srsly                   2.4.6
starlette               0.26.1
svglib                  1.5.1
tb-nightly              2.13.0a20230422
tensorboard-data-server 0.7.0
tensorboard-plugin-wit  1.8.1
thinc                   8.1.9
tifffile                2023.4.12
timm                    0.6.7
tinycss2                1.2.1
tokenizers              0.12.1
tomli                   2.0.1
toolz                   0.12.0
torch                   1.13.1+cu117
torchdiffeq             0.2.3
torchmetrics            0.11.4
torchsde                0.2.5
torchvision             0.14.1+cu117
tqdm                    4.65.0
trampoline              0.1.2
transformers            4.20.1
typer                   0.7.0
typing_extensions       4.5.0
tzdata                  2023.3
uc-micro-py             1.0.1
urllib3                 1.26.15
uvicorn                 0.21.1
wasabi                  1.1.1
wcwidth                 0.2.6
webencodings            0.5.1
websockets              11.0.2
Werkzeug                2.2.3
wheel                   0.40.0
xformers                0.0.16rc425
yapf                    0.33.0
yarl                    1.9.1

janwilmans commented 1 year ago

these are the packages that you have different version of, compared to mine:

aenum                        3.1.11
albumentations               1.3.0
astunparse                   1.6.3
attrs                        22.2.0
beautifulsoup4               4.11.2
bitsandbytes                 0.35.0
charset-normalizer           3.0.1
diffusers                    0.10.2
discord-webhook              1.0.0
facexlib                     0.2.5
fastapi                      0.90.1
ffmpy                        0.3.0
filelock                     3.9.0
flatbuffers                  23.3.3
fonttools                    4.38.0
fsspec                       2023.1.0
gast                         0.4.0
gdown                        4.6.4
GitPython                    3.1.27
google-auth                  2.16.1
google-auth-oauthlib         0.4.6
google-pasta                 0.2.0
gradio                       3.16.2
grpcio                       1.51.3
h5py                         3.8.0
httpx                        0.23.3
huggingface-hub              0.12.1
imageio                      2.26.0
importlib-metadata           6.0.0
joblib                       1.2.0
keras                        2.11.0
lazy_loader                  0.1
libclang                     15.0.6.1
lion-pytorch                 0.0.7
lmdb                         1.4.0
Markdown                     3.4.1
matplotlib                   3.7.0
mdit-py-plugins              0.3.4
mediapipe                    0.9.1.0
modelcards                   0.1.6
multidict                    6.0.4
mypy-extensions              1.0.0
networkx                     3.0
opencv-contrib-python        4.7.0.72
opencv-python-headless       4.7.0.72
opt-einsum                   3.3.0
orjson                       3.8.7
packaging                    23.0
pandas                       1.5.3
protobuf                     3.19.6
psutil                       5.9.4
pyasn1                       0.4.8
pyasn1-modules               0.2.8
pycryptodome                 3.17
pydantic                     1.10.5
pyDeprecate                  0.3.2
pyre-extensions              0.0.23
pytorch-lightning            1.7.6
pytz                         2022.7.1
qudida                       0.0.4
regex                        2022.10.31
requests                     2.25.1
rfc3986                      1.5.0
safetensors                  0.2.7
scikit-learn                 1.2.1
sentencepiece                0.1.97
setuptools                   63.2.0
six                          1.16.0
soupsieve                    2.4
starlette                    0.23.1
tb-nightly                   2.12.0a20230126
tensorboard                  2.11.0
tensorboard-data-server      0.6.1
tensorboard-plugin-wit       1.8.1
tensorflow                   2.11.0
tensorflow-estimator         2.11.0
tensorflow-intel             2.11.0
tensorflow-io-gcs-filesystem 0.31.0
termcolor                    2.2.0
threadpoolctl                3.1.0
tifffile                     2023.2.28
tokenizers                   0.13.2
torchmetrics                 0.11.3
tqdm                         4.64.1
transformers                 4.25.1
typing-inspect               0.8.0
uc-micro-py                  1.0.1
urllib3                      1.26.14
uvicorn                      0.20.0
websockets                   10.4
wheel                        0.38.4
wrapt                        1.15.0
xformers                     0.0.17.dev464
yapf                         0.32.0
yarl                         1.8.2
zipp                         3.15.0

robproctor commented 1 year ago

@janwilmans Good luck, I wouldn't know where to start. I wasn't really paying attention when I installed SD stuff, but it was approximately around 3/5/23 which was when I made a backup of the installation. I think maybe I did some updates once for dreambooth and installed xformers, but that was about it.

Jhonthe7th commented 1 year ago

@robproctor I was using the revAnimated Checkpoint: revAnimated_v122.safetensors [4199bcdd14] without VAE, though now that I think about it, that checkpoint might have one backed in.

Changed to chilloutmix_NiPrunedFp32Fix.safetensors [fc2511737a] as you had and still same issue. It is slightly less apparent in your picture, but the face is much better in full quality preview before completion. (Left is zoomed preview, right is final piece, matching your results)

This is the full output of my prompt script from above for this checkpoint:

Jhonthe7th commented 1 year ago

Tested a little more.

The culprit seems to be around the "denoising" happening in sd_samplers_kdiffusion.py .

If we provide negative prompts, we generate pictures for positive-prompt, negative-prompt and combined (denoised). You can switch the live preview in settings to either.

We can see that independantly, positive and negative promts generate good images over time whilst slowly converging. However the combination or "denoised" picture is garbage. (also visible in live preview if set to in settings)

I output all three for each step and put them in a grid. PositivePrompt | NegativePrompt | Combined (denoised)

Only doubt I still have for this being the problem is the fact that in the last step all three pictures are already the same and broken before the denoising code "combine_denoised()" is called.

Denoising_problem

robproctor commented 1 year ago

@Jhonthe7th Thanks for clarifying, and now I see that I am suffering the same ill-fated issue.

Doing some additional testing, things I found:

Negative prompts increase the effect
Negative embeddings greatly increase the effect
The only embedding which had the seemingly least breaking was easynegative.safetensors which is also the only st one I have, related? I can't find any other st embeds.
The smaller the subject, the more broken. 512x512 results in more often broken. 768x768 results in less often broken.
Under 30 samples more broken. Over 30 samples less broken.
DPM++ 2S a Karras not broken (but likely because of the double steps)
Settings > Sampler Parameters > eta for ancestral samplers - anything greater than 0 has notice broken. 0 has much less broken, still noticeable burn.
Affects many models, tested most popular models.
Some seeds work better than others. Seems more dependent on the size of the subject than anything.

janwilmans commented 1 year ago

@robproctor can you tell me what to change to get your output of all the series of images in all stages?

janwilmans commented 1 year ago

The culprit seems to be around the "denoising" happening in sd_samplers_kdiffusion.py .

Thank you @Jhonthe7th, the kind of the mosaic effect seen in the denoised image is exactly the kind of distortion, that is integrated into the final image! However, I'm afraid I do not how to fix it yet, but I'm certainly going to try!

robproctor commented 1 year ago

@janwilmans You can use this to save the images, there are others as well:

https://github.com/AlUlkesh/sd_save_intermediate_images

It's very confusing as to what exactly is going on here. I have read through every post I can find about it, tried modifying ksampler script, tried various scripts/extensions (antiburn, save last step, etc), but none of them seem to actually work as intended. So, I am really at a loss here. The only solution I have is to turn on combined preview with a higher step count and pretend it is not there :)

robproctor commented 1 year ago

One more thing, clip skip 1 on the chilloutmix example always results in a bad image, seemingly no matter what. The quality of the result seems better with less stuff going on in the background.

On clip skip 2 the broken effect is reduced drastically. The over shapen is still happening, but i seems less. On closeups the brokenness is practically gone, but you still see the high jump between the final combined render and the prompt preview.

janwilmans commented 1 year ago

retested the superman ring on latest master (tagged 1.1.1) (hash 5ab7f213bec2f816f9c5644becb32eb72c8ffb89)

superman ring
Steps: 10, Sampler: LMS, CFG scale: 7, Seed: 42, Size: 512x512, Model hash: 6ce0161689, Model: v1-5-pruned-emaonly

Conclusion:

new xformers 1.17 and torch 2.x versions do not effect the problem

robproctor commented 1 year ago

@janwilmans Interesting to see how different the superman ring is from the updates you made.

I've tried playing around with the k diffusion a bit, but I don't know if the problem is there. Maybe I am misunderstanding, but if LMS produces a broken superman ring and LMS Karras works, would it not be related to the scheduler? From what it seems anything using the new karras algorithm for the scheduler does not appear to have the broken problem. Right? Or no?

LMS 00536-42

LMS Karras 00532-42

Kamekos commented 1 year ago

--medvram & disabling realtime image previews fixed it for me

robproctor commented 1 year ago

@janwilmans I think you may have used the epicDiffusion11 model for the superman ring in your last render and not v1-5-pruned-emaonly as noted in the generation, which would explain why it looked so different.

@Apatiste Thanks but that did not work for me. Can you try and render the superman ring and see if you get the broken ring as shown in the images? It's using the standard SD 1.5 model, v1-5-pruned-emaonly you can just drag the image from github onto your PNG Info image area within the WebUI and then send the generation data to txt2img, Make sure you are using LMS for the sampler and not LMS Karras.

janwilmans commented 1 year ago

--medvram & disabling realtime image previews fixed it for me

can you generate the superman ring with and without your fix and post the result ?

janwilmans commented 1 year ago

@robproctor you are right, I used by accident, which was the non-emaonly version with a different model_hash (of course)

janwilmans commented 1 year ago

@Jhonthe7th can you show me how to generate the images for the three stages you posted?

Jhonthe7th commented 1 year ago

@janwilmans Here you go

Add these into your ..\modules\sd_samplers_kdiffusion.py

New list in your CFGDenoiser for the output image if you want it as a grid. I am appending the states each step. Otherwise just adjust as you need in the bottom code.

   self.output_img = []

In the forward() function before the "return denoised" statement is where I added following code to intercept and save. Currently it outputs incoming picture(x_in) + denoised prompt img+ denoised neg.prompt img + combined denoised

    pics = [ sd_samplers_common.sample_to_image(x_in[0:-uncond.shape[0]]), \
             sd_samplers_common.sample_to_image(x_out[0:-uncond.shape[0]]),\
             sd_samplers_common.sample_to_image(x_out[-uncond.shape[0]:]), \
             sd_samplers_common.sample_to_image(denoised)]

    w, h = zip(*(i.size for i in pics))
    new_pic = Image.new('RGB', (sum(w),max(h)))
    x_offset = 0
    for pic in pics:
        new_pic.paste(pic, (x_offset,0))
        x_offset += pic.size[0]

    self.output_img.append(new_pic)

    if self.step == state.sampling_steps:
        grid = images.image_grid(self.output_img, 1, state.sampling_steps)
        grid.info["parameters"] = "Bug hunting on denoising code"
        images.save_image(grid, "outputs/txt2img-images", "", suffix="-overview"+str(self.step))

They will be saved in your default txt2img output folder amongst the final picture.

Jhonthe7th commented 1 year ago

@janwilmans Ah, one heads-up I forgot. The "sample_to_image" uses the "live-preview" settings for scaling. So if you have it set to ApproxNN, then also output pictures are small. I set it to full quality and disabled the live-preview to get the HQ-output I posted before.

robproctor commented 1 year ago

@Jhonthe7th Thanks for the script, that will definitely come in handy!

Adam-Burke commented 1 year ago

@janwilmans Interesting to see how different the superman ring is from the updates you made.

I've tried playing around with the k diffusion a bit, but I don't know if the problem is there. Maybe I am misunderstanding, but if LMS produces a broken superman ring and LMS Karras works, would it not be related to the scheduler? From what it seems anything using the new karras algorithm for the scheduler does not appear to have the broken problem. Right? Or no?

LMS

LMS Karras

I get the same results here with LMS Karras. It seems to work. That's the only one i've tried that works so far, including those suggested above.

janwilmans commented 1 year ago

@Jhonthe7th I tried you code, but "images" is not part of forward(); on 22bcc7be428c94e9408f589966c2040187245d81, which I reverted to since 5ab7f21 (current head) is broken.

this is my (non-working) class

class CFGDenoiser(torch.nn.Module):
    """
    Classifier free guidance denoiser. A wrapper for stable diffusion model (specifically for unet)
    that can take a noisy picture and produce a noise-free picture using two guidances (prompts)
    instead of one. Originally, the second prompt is just an empty string, but we use non-empty
    negative prompt.
    """

    def __init__(self, model):
        super().__init__()
        self.inner_model = model
        self.mask = None
        self.nmask = None
        self.init_latent = None
        self.step = 0
        self.image_cfg_scale = None
        self.output_img = []

    def combine_denoised(self, x_out, conds_list, uncond, cond_scale):
        denoised_uncond = x_out[-uncond.shape[0]:]
        denoised = torch.clone(denoised_uncond)

        for i, conds in enumerate(conds_list):
            for cond_index, weight in conds:
                denoised[i] += (x_out[cond_index] - denoised_uncond[i]) * (weight * cond_scale)

        return denoised

    def combine_denoised_for_edit_model(self, x_out, cond_scale):
        out_cond, out_img_cond, out_uncond = x_out.chunk(3)
        denoised = out_uncond + cond_scale * (out_cond - out_img_cond) + self.image_cfg_scale * (out_img_cond - out_uncond)

        return denoised

    def forward(self, x, sigma, uncond, cond, cond_scale, image_cond):
        if state.interrupted or state.skipped:
            raise sd_samplers_common.InterruptedException

        # at self.image_cfg_scale == 1.0 produced results for edit model are the same as with normal sampling,
        # so is_edit_model is set to False to support AND composition.
        is_edit_model = shared.sd_model.cond_stage_key == "edit" and self.image_cfg_scale is not None and self.image_cfg_scale != 1.0

        conds_list, tensor = prompt_parser.reconstruct_multicond_batch(cond, self.step)
        uncond = prompt_parser.reconstruct_cond_batch(uncond, self.step)

        assert not is_edit_model or all([len(conds) == 1 for conds in conds_list]), "AND is not supported for InstructPix2Pix checkpoint (unless using Image CFG scale = 1.0)"

        batch_size = len(conds_list)
        repeats = [len(conds_list[i]) for i in range(batch_size)]

        if shared.sd_model.model.conditioning_key == "crossattn-adm":
            image_uncond = torch.zeros_like(image_cond)
            make_condition_dict = lambda c_crossattn, c_adm: {"c_crossattn": c_crossattn, "c_adm": c_adm} 
        else:
            image_uncond = image_cond
            make_condition_dict = lambda c_crossattn, c_concat: {"c_crossattn": c_crossattn, "c_concat": [c_concat]} 

        if not is_edit_model:
            x_in = torch.cat([torch.stack([x[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [x])
            sigma_in = torch.cat([torch.stack([sigma[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [sigma])
            image_cond_in = torch.cat([torch.stack([image_cond[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [image_uncond])
        else:
            x_in = torch.cat([torch.stack([x[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [x] + [x])
            sigma_in = torch.cat([torch.stack([sigma[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [sigma] + [sigma])
            image_cond_in = torch.cat([torch.stack([image_cond[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [image_uncond] + [torch.zeros_like(self.init_latent)])

        denoiser_params = CFGDenoiserParams(x_in, image_cond_in, sigma_in, state.sampling_step, state.sampling_steps, tensor, uncond)
        cfg_denoiser_callback(denoiser_params)
        x_in = denoiser_params.x
        image_cond_in = denoiser_params.image_cond
        sigma_in = denoiser_params.sigma
        tensor = denoiser_params.text_cond
        uncond = denoiser_params.text_uncond

        if tensor.shape[1] == uncond.shape[1]:
            if not is_edit_model:
                cond_in = torch.cat([tensor, uncond])
            else:
                cond_in = torch.cat([tensor, uncond, uncond])

            if shared.batch_cond_uncond:
                x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict([cond_in], image_cond_in))
            else:
                x_out = torch.zeros_like(x_in)
                for batch_offset in range(0, x_out.shape[0], batch_size):
                    a = batch_offset
                    b = a + batch_size
                    x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict([cond_in[a:b]], image_cond_in[a:b]))
        else:
            x_out = torch.zeros_like(x_in)
            batch_size = batch_size*2 if shared.batch_cond_uncond else batch_size
            for batch_offset in range(0, tensor.shape[0], batch_size):
                a = batch_offset
                b = min(a + batch_size, tensor.shape[0])

                if not is_edit_model:
                    c_crossattn = [tensor[a:b]]
                else:
                    c_crossattn = torch.cat([tensor[a:b]], uncond)

                x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict(c_crossattn, image_cond_in[a:b]))

            x_out[-uncond.shape[0]:] = self.inner_model(x_in[-uncond.shape[0]:], sigma_in[-uncond.shape[0]:], cond=make_condition_dict([uncond], image_cond_in[-uncond.shape[0]:]))

        denoised_params = CFGDenoisedParams(x_out, state.sampling_step, state.sampling_steps)
        cfg_denoised_callback(denoised_params)

        devices.test_for_nans(x_out, "unet")

        if opts.live_preview_content == "Prompt":
            sd_samplers_common.store_latent(x_out[0:uncond.shape[0]])
        elif opts.live_preview_content == "Negative prompt":
            sd_samplers_common.store_latent(x_out[-uncond.shape[0]:])

        if not is_edit_model:
            denoised = self.combine_denoised(x_out, conds_list, uncond, cond_scale)
        else:
            denoised = self.combine_denoised_for_edit_model(x_out, cond_scale)

        if self.mask is not None:
            denoised = self.init_latent * self.mask + self.nmask * denoised

        self.step += 1

        pics = [ sd_samplers_common.sample_to_image(x_in[0:-uncond.shape[0]]), 
                    sd_samplers_common.sample_to_image(x_out[0:-uncond.shape[0]]),
                    sd_samplers_common.sample_to_image(x_out[-uncond.shape[0]:]), 
                    sd_samplers_common.sample_to_image(denoised)]

        w, h = zip(*(i.size for i in pics))
        new_pic = Image.new('RGB', (sum(w),max(h)))
        x_offset = 0
        for pic in pics:
            new_pic.paste(pic, (x_offset,0))
            x_offset += pic.size[0]

        self.output_img.append(new_pic)

        if self.step == state.sampling_steps:
            grid = images.image_grid(self.output_img, 1, state.sampling_steps)    # here "images" does not exist
            grid.info["parameters"] = "Bug hunting on denoising code"
            images.save_image(grid, "outputs/txt2img-images", "", suffix="-overview"+str(self.step))

        return denoised

robproctor commented 1 year ago

@janwilmans I think he forgot to include the imports, you need to also import the images module and the PIL image,

At the top of the k diffusion file where you have the from from modules line

from modules import prompt_parser, devices, sd_samplers_common

Include the images module at the end

from modules import prompt_parser, devices, sd_samplers_common, images

Additionally you will need to add a new line to import the image library

from PIL import Image

That was what I did to get it to work.

Thanks

janwilmans commented 1 year ago

Thanks, that works!

What are the "kind of image" we seen here, left to right @Jhonthe7th ?

janwilmans commented 1 year ago

Very clear example on Euler a

masterpiece,cowboy pretty face, nice sweater, sfw
Negative prompt: (bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5,(NG_DeepNegative_V1_75T:1.3)
Steps: 10, Sampler: Euler a, CFG scale: 8, Seed: 901305577, Size: 512x512, Model hash: e714ee20aa, Model: a_abyssorangemix2_Hard

The face and eyes are good on the second-to-last-row (middle two images), but then on the last row, the face and eyes are all messed up.

It looks like the last last from the second-to-last row, is almost (completely?) identical to the images on the last low

robproctor commented 1 year ago

@janwilmans from what @Jhonthe7th said the grid is composed of:

"Currently it outputs incoming picture(x_in) + denoised prompt img+ denoised neg.prompt img + combined denoised"

x_in is the combined picture + the noise from the scheduler before being denoised again.

Looking at your pictures, I think perhaps the issue is the combining of the denoised images and not the denoising, but I could be wrong. In an earlier message you noted that you updated to 1.1.1, looking through the code of 1.1+ there is significant changes to the sd_samplers_kdiffusion.py script, notably around combining denoised images. Do you know if the k sampler was updated in your commit already?

I am too newb to figure out how to compare the timeline of commits to determine the answer, and too newb with venv to easily update and go back to working version. If you can easily test the last 1.1 version it may be worth a shot. Though, those edits may have already been in your commit, I am not sure how to determine.

https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/3591eefedfc1f4dc324710b3d8e532cf73572852#diff-c4515db9d705a6ba40c2b7756690c75c2e1dd2c8edc3e04930a86c1f06ce7d20

hashborgir commented 1 year ago

Hi guys. Please have a look at my latest comment here: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10223#issuecomment-1548932075

It seems that either A1111 or SD code is somehow causing corruption at the last save step, however, all the steps saved in the /steps/ directory during generation using this script: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Custom-Scripts/d7ddc139ff8e1887eac4b280be4b566b7ead5799#saving-steps-of-the-sampling-process seem fine, even the last saved step. It's just whatever it finally output into /outputs/ directory that is corrupted.

AUTOMATIC1111 / stable-diffusion-webui