Closed iamrohitanshu closed 1 year ago
I'm seeing a similar thing off any Controlnet, not just shuffle.
I also have issues with Adetailer+ControlNET. In my case when I try to use both of them I get the following error: ADetailer: 'NansException' occurred with 1st settings. A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check. I am using ---xformers --no-half-vae, so I shouldn't be getting NansExceptions. I didn't use to have any issues, but after the new update a few days ago, this started happening. Haven't found any way to solve this on my own.
Here is my result with Controlnet shuffle and ADetailer:
python launch.py --autolaunch --xformers --disable-nan-check
Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:34:57) [MSC v.1936 64 bit (AMD64)]
Version: v1.4.0-101-gb42c0ef6
Commit hash: b42c0ef6c31e38db52aecdb38908238dc81c5f01
Installing requirements
Launching Web UI with arguments: --autolaunch --xformers --disable-nan-check
[-] ADetailer initialized. version: 23.7.0, num models: 8
2023-07-02 23:41:54,179 - ControlNet - INFO - ControlNet v1.1.227
ControlNet preprocessor location: D:\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-07-02 23:41:54,379 - ControlNet - INFO - ControlNet v1.1.227
Loading weights [5998292c04] from D:\stable-diffusion-webui\models\Stable-diffusion\Counterfeit-V3.0_fp16-no-ema.safetensors
Creating model from config: D:\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
DiffusionWrapper has 859.52 M params.
Startup time: 13.9s (import torch: 2.9s, import gradio: 2.8s, import ldm: 0.9s, other imports: 1.5s, setup codeformer: 0.1s, list SD models: 0.4s, load scripts: 3.9s, create ui: 1.0s, gradio launch: 0.4s).
preload_extensions_git_metadata for 16 extensions took 5.46s
Loading VAE weights specified in settings: D:\stable-diffusion-webui\models\VAE\kl-f8-anime2.safetensors
Applying attention optimization: xformers... done.
Textual inversion embeddings loaded(6): bad_prompt, bad_prompt_version2, badhandv4, EasyNegative, EasyNegativeV2, negative_hand-neg
Model loaded in 12.3s (load weights from disk: 1.1s, create model: 0.8s, apply weights to model: 5.8s, apply half(): 2.1s, load VAE: 0.7s, move model to device: 1.5s, calculate empty prompt: 0.2s).
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 2.92it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.40it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.68it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00, 3.87it/s]
0: 640x448 1 face, 81.6ms
Speed: 5.0ms preprocess, 81.6ms inference, 9.7ms postprocess per image at shape (1, 3, 640, 448)
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.57it/s]
0: 640x448 1 face, 7.0ms
Speed: 2.0ms preprocess, 7.0ms inference, 2.0ms postprocess per image at shape (1, 3, 640, 448)
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.94it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:13<00:00, 1.51it/s]
2023-07-02 23:43:52,640 - ControlNet - INFO - Loading model: control_v11e_sd15_shuffle [526bfdae]0:13<00:00, 3.87it/s]
2023-07-02 23:43:53,902 - ControlNet - INFO - Loaded state_dict from [D:\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11e_sd15_shuffle.pth]
2023-07-02 23:43:53,903 - ControlNet - INFO - Loading config: D:\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11e_sd15_shuffle.yaml
2023-07-02 23:43:56,234 - ControlNet - INFO - ControlNet model control_v11e_sd15_shuffle [526bfdae] loaded.
2023-07-02 23:43:56,343 - ControlNet - INFO - Loading preprocessor: shuffle
2023-07-02 23:43:56,344 - ControlNet - INFO - preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:07<00:00, 2.74it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 2.79it/s]
0: 640x448 1 face, 13.0ms
Speed: 1.0ms preprocess, 13.0ms inference, 1.0ms postprocess per image at shape (1, 3, 640, 448)
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.96it/s]
0: 640x448 1 face, 7.1ms
Speed: 1.3ms preprocess, 7.1ms inference, 1.0ms postprocess per image at shape (1, 3, 640, 448)
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.94it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:14<00:00, 1.41it/s]
2023-07-02 23:44:22,868 - ControlNet - INFO - Loading model from cache: control_v11e_sd15_shuffle [526bfdae] 2.79it/s]
2023-07-02 23:44:22,871 - ControlNet - INFO - Loading preprocessor: shuffle
2023-07-02 23:44:22,872 - ControlNet - INFO - preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:07<00:00, 2.77it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 2.83it/s]
0: 640x448 1 face, 6.8ms
Speed: 2.0ms preprocess, 6.8ms inference, 1.5ms postprocess per image at shape (1, 3, 640, 448)
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.98it/s]
0: 640x448 1 face, 6.0ms
Speed: 2.0ms preprocess, 6.0ms inference, 2.4ms postprocess per image at shape (1, 3, 640, 448)
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.98it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:14<00:00, 1.37it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:14<00:00, 2.83it/s]
There are a lot of things involved, so it's hard to know what's causing it. It may not be a problem with controlnet and adetailer. I've even seen posts where rebooting the computer solved a similar problem...
Yes, it’s very inconsistent on my end as well. I’ll do it in t2i and then i2i and it’ll work in one and not the other. I did just update my automatic1111 repo yesterday so that does seem like a likely culprit.
On Sun, Jul 2, 2023 at 10:49 AM Dowon @.***> wrote:
Here is my result with Controlnet shuffle and ADetailer:
[image: Cap 2023-07-02 23-45-06-705] https://user-images.githubusercontent.com/37621276/250364383-87fa4435-ce14-4d71-ac77-aa181fa31582.png
python launch.py --autolaunch --xformers --disable-nan-check Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:34:57) [MSC v.1936 64 bit (AMD64)] Version: v1.4.0-101-gb42c0ef6 Commit hash: b42c0ef6c31e38db52aecdb38908238dc81c5f01 Installing requirements
Launching Web UI with arguments: --autolaunch --xformers --disable-nan-check [-] ADetailer initialized. version: 23.7.0, num models: 8 2023-07-02 23:41:54,179 - ControlNet - INFO - ControlNet v1.1.227 ControlNet preprocessor location: D:\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads 2023-07-02 23:41:54,379 - ControlNet - INFO - ControlNet v1.1.227 Loading weights [5998292c04] from D:\stable-diffusion-webui\models\Stable-diffusion\Counterfeit-V3.0_fp16-no-ema.safetensors Creating model from config: D:\stable-diffusion-webui\configs\v1-inference.yaml LatentDiffusion: Running in eps-prediction mode Running on local URL: http://127.0.0.1:7860
To create a public link, set
share=True
inlaunch()
.DiffusionWrapper has 859.52 M params.Startup time: 13.9s (import torch: 2.9s, import gradio: 2.8s, import ldm: 0.9s, other imports: 1.5s, setup codeformer: 0.1s, list SD models: 0.4s, load scripts: 3.9s, create ui: 1.0s, gradio launch: 0.4s).preload_extensions_git_metadata for 16 extensions took 5.46sLoading VAE weights specified in settings: D:\stable-diffusion-webui\models\VAE\kl-f8-anime2.safetensorsApplying attention optimization: xformers... done.Textual inversion embeddings loaded(6): bad_prompt, bad_prompt_version2, badhandv4, EasyNegative, EasyNegativeV2, negative_hand-negModel loaded in 12.3s (load weights from disk: 1.1s, create model: 0.8s, apply weights to model: 5.8s, apply half(): 2.1s, load VAE: 0.7s, move model to device: 1.5s, calculate empty prompt: 0.2s).100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 2.92it/s]Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.40it/s]100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.68it/s]Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00, 3.87it/s]0: 640x448 1 face, 81.6msSpeed: 5.0ms preprocess, 81.6ms inference, 9.7ms postprocess per image at shape (1, 3, 640, 448)100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.57it/s]0: 640x448 1 face, 7.0msSpeed: 2.0ms preprocess, 7.0ms inference, 2.0ms postprocess per image at shape (1, 3, 640, 448)100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.94it/s]Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:13<00:00, 1.51it/s]2023-07-02 23:43:52,640 - ControlNet - INFO - Loading model: control_v11e_sd15_shuffle [526bfdae]0:13<00:00, 3.87it/s]2023-07-02 23:43:53,902 - ControlNet - INFO - Loaded state_dict from [D:\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11e_sd15_shuffle.pth]2023-07-02 23:43:53,903 - ControlNet - INFO - Loading config: D:\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11e_sd15_shuffle.yaml2023-07-02 23:43:56,234 - ControlNet - INFO - ControlNet model control_v11e_sd15_shuffle [526bfdae] loaded.2023-07-02 23:43:56,343 - ControlNet - INFO - Loading preprocessor: shuffle2023-07-02 23:43:56,344 - ControlNet - INFO - preprocessor resolution = 512100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:07<00:00, 2.74it/s]Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 2.79it/s]0: 640x448 1 face, 13.0msSpeed: 1.0ms preprocess, 13.0ms inference, 1.0ms postprocess per image at shape (1, 3, 640, 448)100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.96it/s]0: 640x448 1 face, 7.1msSpeed: 1.3ms preprocess, 7.1ms inference, 1.0ms postprocess per image at shape (1, 3, 640, 448)100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.94it/s]Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:14<00:00, 1.41it/s]2023-07-02 23:44:22,868 - ControlNet - INFO - Loading model from cache: control_v11e_sd15_shuffle [526bfdae] 2.79it/s]2023-07-02 23:44:22,871 - ControlNet - INFO - Loading preprocessor: shuffle2023-07-02 23:44:22,872 - ControlNet - INFO - preprocessor resolution = 512100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:07<00:00, 2.77it/s]Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 2.83it/s]0: 640x448 1 face, 6.8msSpeed: 2.0ms preprocess, 6.8ms inference, 1.5ms postprocess per image at shape (1, 3, 640, 448)100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.98it/s]0: 640x448 1 face, 6.0msSpeed: 2.0ms preprocess, 6.0ms inference, 2.4ms postprocess per image at shape (1, 3, 640, 448)100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:01<00:00, 6.98it/s]Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:14<00:00, 1.37it/s]Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:14<00:00, 2.83it/s]There are a lot of things involved, so it's hard to know what's causing it. It may not be a problem with controlnet and adetailer. I've even seen posts where rebooting the computer solved a similar problem...
— Reply to this email directly, view it on GitHub https://github.com/Bing-su/adetailer/issues/181#issuecomment-1616687710, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB56AC2KTM2MERCEKL7SGQDXOGC7PANCNFSM6AAAAAAZ3MLNRE . You are receiving this because you are subscribed to this thread.Message ID: @.***>
I also have issues with Adetailer+ControlNET. In my case when I try to use both of them I get the following error: ADetailer: 'NansException' occurred with 1st settings. A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check. I am using ---xformers --no-half-vae, so I shouldn't be getting NansExceptions. I didn't use to have any issues, but after the new update a few days ago, this started happening. Haven't found any way to solve this on my own.
I have these arguments in my commandline from day 1 as my video card was producing black images on first try. So, I don't get any errors like that in my console, but this issue is showing in the results I guess.
@Bing-su , @pwcarney So I did more testing. It has been consistent on my system. It glitches every time. May be it has to do something with hardware, or commandline arguments. I tried changing the arguments, but can't change much as they give OOM error as I have only a 6GB VRAM card.
A workaround is to generate without Adetailer and then use it in img2img later without controlnet. We have to do almost twice the work, but it works.
@Bing-su , If you think it's not reproducible or not worth investigating into further, you can close this issue if you'd like. Thank you everyone for their inputs.
@iamrohitanshu Did you encounter this issue after webui 1.4.0? If so, try updating adetailer as I've fixed one related issue.
I also have issues with Adetailer+ControlNET. In my case when I try to use both of them I get the following error: ADetailer: 'NansException' occurred with 1st settings. A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check. I am using ---xformers --no-half-vae, so I shouldn't be getting NansExceptions. I didn't use to have any issues, but after the new update a few days ago, this started happening. Haven't found any way to solve this on my own.
Same error here, tried numerous reasonable solution from previous thread, still doesn't work. And this issue doesn't seem to be taken care.
So, if anyone having the same issue, I suggest using https://github.com/ototadana/sd-face-editor instead.
@iamrohitanshu Did you encounter this issue after webui 1.4.0? If so, try updating adetailer as I've fixed one related issue.
@Bing-su I updated both Adetailer and ControlNet. Unfortunately, same result.
So, if anyone having the same issue, I suggest using https://github.com/ototadana/sd-face-editor instead.
@Gardgio Thanks for the recommendation, looks nice. I'm trying it.
Can anyone tell me what version of webui this is?
That's my personal fork, and the contents are the same as v1.4.0, 394ffa7b0a7fff3ec484bcd084e673a8b301ccc8
.
A week has passed and no one can fix this bug? I have tried different models with different vae's and all have this problem. 2060s, 3070, 4070ti, 4090. Is it a problem with webui or with xformers or something else?
webui1.4 ControlNet+Adetailer
Do I have to go back to 1.3 webui if I want to use it?
@Bing-su , @pwcarney , @xiaojiemeidu I checked. It's working as intended on webui v1.3.2, but messes up on v1.4.0.
i've same problem too when use with controlnet openpose, lineart.
Same issue when using a detailer+ reference controlnets :/
@Bing-su , @pwcarney , @xiaojiemeidu I checked. It's working as intended on webui v1.3.2, but messes up on v1.4.0.
how do you downgrade the webui pls?
how do you downgrade the webui pls?
Instead of downgrading, I've just installed the v1.3.2 separately. So, that I can use all my extensions and settings on the v1.4.0 and receive the upgrades and fixes as they are released.
To install an old version from github, just like when you first installed the webui, use
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
in a separate folder through the command line,
then write
git checkout baf6946
where baf6946
is the hash of the release v1.3.2, which you can check in the releases section of the original repo.
You can just copy the folders of the extensions you need to the new installation, no need to install again.
IMPORTANT: Before you run the webui-user.bat
file, make sure to copy your desired models in their respective folders, otherwise if it's empty, it will automatically download the v1-5-pruned-emaonly.safetensors
file, which is around 4GB. Also, don't use git pull
in the webui-user.bat
file, so that it doesn't auto upgrade.
On v1.3.2, I've only installed the bare minimum of things that I absolutely need. That keeps it light and uncluttered. Hope that helps.
I also have issues with Adetailer+ControlNET. In my case when I try to use both of them I get the following error: ADetailer: 'NansException' occurred with 1st settings. A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check. I am using ---xformers --no-half-vae, so I shouldn't be getting NansExceptions. I didn't use to have any issues, but after the new update a few days ago, this started happening. Haven't found any way to solve this on my own.
got the same problem at last I will return the webui to version 1.3.2 and this problem disappears
how do you downgrade the webui pls?
Instead of downgrading, I've just installed the v1.3.2 separately. So, that I can use all my extensions and settings on the v1.4.0 and receive the upgrades and fixes as they are released.
To install an old version from github, just like when you first installed the webui, use
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
in a separate folder through the command line, then writegit checkout baf6946
wherebaf6946
is the hash of the release v1.3.2, which you can check in the releases section of the original repo.You can just copy the folders of the extensions you need to the new installation, no need to install again.
IMPORTANT: Before you run the
webui-user.bat
file, make sure to copy your desired models in their respective folders, otherwise if it's empty, it will automatically download thev1-5-pruned-emaonly.safetensors
file, which is around 4GB. Also, don't usegit pull
in thewebui-user.bat
file, so that it doesn't auto upgrade.On v1.3.2, I've only installed the bare minimum of things that I absolutely need. That keeps it light and uncluttered. Hope that helps.
Downgrading worked for me as well. Thanks for finding the commit. What a weird issue. I wonder what the cause is. I'm content to just chill on this release for a bit... Until SDXL launches at least.
Can anyone confirm if this issue exists with the latest version of ADetailer and webui 1.4.0?
@Bing-su , just tested, this works as it should now. The faces are clean once again! ╰(°▽°)╯
Though my webui has been upgraded to v1.4.1. So I'm not sure if it works on v1.4.0 or not. Thanks for your hard work!
Downgrading worked for me as well. Thanks for finding the commit. What a weird issue. I wonder what the cause is. I'm content to just chill on this release for a bit... Until SDXL launches at least.
No problem. It's working on v1.4.1, so you might want to check that. SDXL is supposed to be released this month, but it seems to be too big for my graphics card. I'll wait to see if someone prunes it enough that it works on my system or I might've to save money to buy a 12GB model. ¯\(ツ)/¯
The base model is already good enough, I'm excited to see what model creators and mergers bring to the table. (❁´◡`❁)
same here
@Sotonya , Did you update the extension. I'm the original poster of this issue, and it works fine now for me. Please update the extension and report back either way, so we can be sure.
the fact is that I have this problem appeared exactly after the update. But I'll try again, thanks. Upd.
Updated to 23.7.5 for now, and nothing changed. face is still messy
@Sotonya what's your webui version? is it updated to v1.4.1 as well?
@Sotonya what's your webui version? is it updated to v1.4.1 as well?
Yes
@Sotonya Well, only thing's different is that I have xformers activated. I'd suggest you do that, even just for testing purposes.
Then @Bing-su can take from there.
It worked without xformers. maybe the problem is in colab? I don't use the sd locally.
It worked without xformers. maybe the problem is in colab? I don't use the sd locally.
I use it locally and have never used colab before. So, I can't help you there. :|
@Sotonya Hey if your problem isn't resolved yet, may be you can try some other extension meanwhile, for fixing faces. https://github.com/ototadana/sd-face-editor has worked for me.
Describe the bug
When I use Adetailer or ControlNet shuffle separately, everything works just fine. But using them together is messing up the face, may be it's trying to apply the shufle controlnet to the face, idk though.
As Adetailer is working on the image last, i.e. after controlnet has done it's work, I'm posting this as a bug here. There are no errors or warnings in the console. That just seems normal.
I'm using the model Rev_Animated in this particular instance, but I've checked other models and they are producing the same results as well.
ControlNet in the Adetailer settings is set to None. I even changed the weight to 0 just in case, but it's still the same.
The entire console log, including python version information, webui version, commit hash, commandline arguments and error traces.
List of installed extensions
No response