search

AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
142.48k stars 26.87k forks source link

[Bug]: My PC reboots when I activate Hires.fix #10577

Open kamerosoul opened 1 year ago

kamerosoul commented 1 year ago

Is there an existing issue for this?

What happened?

My PC reboots when I activate Hires.fix My SD runs excellent for generating for text2image or img2img and many extensions. But in txt2img, when activating Hires.fix with any upscaler or option, when click in Generate, my PC reboots immediately. I have a PC HP Z640 64RAM Graphics Card RTX 3080 Ti. Latest video driver, 531.61-desktop-win10-win11-64bit-international-nsd-dch-whql If I send Text2img picture and send to extras, I can upscale correctly without problems. Video RAM runs at the half capacity in text2img 512x768

Steps to reproduce the problem

Checkpoint revAnimated_v11.safetensors [d725be5d18] VAE vae-ft-mse-840000-ema-pruned.safetensors Noise multiplier for img2img = 1 DPM++SDE Steps = 20 512x768 CFG Scale = 6 It generates an excellent picture in 2 seconds But if same setting I select Hires.fix, my PC reboots

What should have happened?

Generate an upcaled picture without rebooting PC

Commit where the problem happens

version: v1.2.1  •  python: 3.10.4  •  torch: 2.0.1+cu118  •  xformers: N/A  •  gradio: 3.29.0  •  checkpoint: d725be5d18

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

--api

List of extensions

Video_loopback_for_webui canvas-zoom deforum-for-automatic1111-webui openOutpaint-webUI-extension openpose-editor sd-webui-ar sd-webui-controlnet

Console logs

G:\Stable_Diffusion\stable-diffusion-webui>git pull
Already up to date.
venv "G:\Stable_Diffusion\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)]
Version: v1.2.1
Commit hash: 89f9faa63388756314e8a1d96cf86bf5e0663045
Installing requirements

No module 'xformers'. Proceeding without it.

Launching Web UI with arguments: --api
No module 'xformers'. Proceeding without it.
ControlNet v1.1.173
ControlNet v1.1.173
Loading weights [d725be5d18] from G:\Stable_Diffusion\stable-diffusion-webui\models\Stable-diffusion\revAnimated_v11.safetensors
*Deforum ControlNet support: enabled*
Creating model from config: G:\Stable_Diffusion\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 40.7s (import torch: 2.9s, import gradio: 2.7s, import ldm: 2.9s, other imports: 9.2s, list SD models: 1.1s, setup codeformer: 1.0s, load scripts: 8.0s, reload hypernets: 0.1s, create ui: 9.9s, gradio launch: 1.4s, scripts app_started_callback: 1.1s).
Loading VAE weights specified in settings: G:\Stable_Diffusion\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(0):
Model loaded in 377.2s (load weights from disk: 5.3s, load config: 0.1s, create model: 1.9s, apply weights to model: 338.9s, apply half(): 1.2s, load VAE: 14.5s, move model to device: 1.3s, load textual inversion embeddings: 13.9s).
100%|█████████████████████████████████████████████████████████████████████| 20/20 [00:22<00:00,  1.14s/it]
Total progress: 20it [06:16, 18.81s/it]
100%|█████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.83it/s]
Total progress: 100%|█████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.73it/s]
100%|█████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.69it/s]
Total progress: 100%|█████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.61it/s]
100%|█████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.43it/s]
Total progress: 100%|█████████████████████████████████████████████████████| 20/20 [00:06<00:00,  3.32it/s]
100%|█████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.76it/s]
Total progress: 100%|█████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.64it/s]
Total progress: 100%|█████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.80it/s]

Additional information

Only Hires.fix has an issue, all other features in SD and extension work fine. I have a PC HP Z640 64RAM Graphics Card RTX 3080 Ti. Latest video driver, 531.61-desktop-win10-win11-64bit-international-nsd-dch-whql

missionfloyd commented 1 year ago

How's your PSU? If the GPU's drawing too much power, it'll trip the overcurrent protection.

kornerson commented 1 year ago

thats your PSU for sure. Been there.

kamerosoul commented 1 year ago

When I recently installed, PSU activate fans while generating image with hires. After 2 weeks and when I realized hires caused it, It just reboots immediately when generating first image. So, hires requires too much power ? Why upscale works fine in extras ?

Kadah commented 1 year ago

Upscale on its own for a single image is pretty light and short operation. Highres fix is generate, upscale, and generate again. Its possible you're tripping one of the time-based over current protections (OCP) of the PSU.

If you have some means to overclock your GPU, try lower the power limit like 5% and see if that stops the issue.

kamerosoul commented 1 year ago

I need to learn how can overclock. Another option is using another PSU with more power ?

kornerson commented 1 year ago

Don't know. But it's the PSU. Had a computer with same problem. Also training a lora caused reboots. Changed the PSU and now it's Stable after weeks

El dom, 21 may 2023, 0:05, kamerosoul @.***> escribió:

When I recently installed, PSU activate fans. After 2 weeks and when I realized hires caused it, It just reboots immediately when generating first image. So, hires requires too much power ? Why upscale works fine in extras ?

— Reply to this email directly, view it on GitHub https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10577#issuecomment-1556025540, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEW7XG3CR6ETU3DWI5LY63XHE5ZDANCNFSM6AAAAAAYI3ODPI . You are receiving this because you commented.Message ID: @.***>

kornerson commented 1 year ago

Overclocking would make it worse. Do you mean underclock?

El El dom, 21 may 2023 a las 1:32, kamerosoul @.***> escribió:

I need to learn how can overclock. Another option is using another PSU with more power ?

— Reply to this email directly, view it on GitHub https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10577#issuecomment-1556042395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEW7XCC3JXW6HJYLJS2SYDXHFIBLANCNFSM6AAAAAAYI3ODPI . You are receiving this because you commented.Message ID: @.***>

EfourC commented 1 year ago

Overclocking would make it worse. Do you mean underclock?

I think he just means general familiarity with overclocking (or underclocking). Specifically, he's suggesting only to reduce the power limit in this case.

kamerosoul commented 1 year ago

Is there any option in SD to reduce operation load in this function? Or an option to do hires in steps: first generate, then upscale then generate?

EfourC commented 1 year ago

Is there any option in SD to reduce operation load in this function? Or an option to do hires in steps: first generate, then upscale then generate?

Those steps are already technically happening one-after-another. There's no option about limiting GPU load, but maybe you could try different optimizations and see if there is a difference. For example try with --xformers.

Otherwise, as a test, you could try disabling Hires fix and make a base image that's the same size that Hires would be. I assume you're leaving the multiplier at x2, so try making an image at 1024x1536 and see if you have the same problem.

You could also try using the img2img tab, input whatever 1024x1536 base image, and then generate an image from it at the same resolution.

I really recommend installing MSI Afterburner. After learning how to use it, you can monitor how much power your card is drawing during different types of activity and get a better feel for how hard your GPU is getting pushed (leading to spikes of power drawn from your PSU). You can also use MSI Afterburner to adjust your power limit.

BTW, what is your PSU's model and rated watts? I agree with the others that your PSU may be the culprit.. if I remember right, I heard the high end of RTX 30 series demand really heavy sudden spikes of current, which even some good PSUs couldn't properly supply. (I could be mistaken about this.)

Edit: Another question.. how far did the Hires fix image get before your PC rebooted? Reboot right away, or did some of the steps complete?

Edit2: MSI Afterburner screenshot

SUOany3YAh

kornerson commented 1 year ago

I had a 800w PSU and had reboots. Changed it to a good 1000w PSU and all reboots are gone.

El dom, 21 may 2023, 9:08, EfourC @.***> escribió:

Is there any option in SD to reduce operation load in this function? Or an option to do hires in steps: first generate, then upscale then generate?

Those steps are already technically happening one-after-another.

As a test, you could try disabling Hires fix and make a base image that's the same size that Hires would be. I assume you're leaving the multiplier at x2, so try making an image at 1024x1536 and see if you have the same problem.

You could also try using the img2img tab, input whatever 1024x1536 base image, and then generate an image from it at the same resolution.

I really recommend installing MSI Afterburner. After learning how to use it, you can monitor how much power your card is drawing during different types of activity and get a better feel for how hard your GPU is getting pushed (leading to spikes of power drawn from your PSU). You can also use MSI Afterburner to adjust your power limit.

BTW, what is your PSU's model and rated watts? I agree with the others that your PSU may be the culprit.. if I remember right, I heard the high end of RTX 30 series demand really heavy sudden spikes of current, which even some good PSUs couldn't properly supply. (I could be mistaken about this.)

— Reply to this email directly, view it on GitHub https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10577#issuecomment-1556103104, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACEW7XHDNKE7OOLZKGKMTADXHG5QNANCNFSM6AAAAAAYI3ODPI . You are receiving this because you commented.Message ID: @.***>

kamerosoul commented 1 year ago

EfourC, thank you for your detailed comments. I will try MSI Afterburner. Actually I can miss hires function. The only problem is that I have a collection of SD images that are good samples. When I want an style, select one of them, drop in PNG info and send to txt2img then generate a new similar sample. But if I forget to review hires fix option and disable it, my pc reboots and loss whatever I have and I am worry if I can damage my PC. I have this information from my PSU: 925W 90% Efficient, Custom PSU (Wide Ranging, Active PFC) Operating Voltage: 90–269 VAC how far did the Hires fix image get before your PC rebooted? Less than 1 second

kamerosoul commented 1 year ago

Thanks, seems I have to do the same

jmp909 commented 1 year ago

common issue with 30xx series: voltage spikes

different nvidia drivers seem to behave differently

as EfourC mentions, knock down your power target in Afterburner

my 3080 FE + Corsair 850W PSU has the same issue eg when trying to play Warzone in 4K or putting heavy load in Stable Diffusion.. I'm looking to upgrade to 1000W with better transient response.

Baughn commented 1 year ago

Different PSUs have dramatically different reactions to the spikes. I'm running a 4090 with a 700W PSU, which works fine...

The spikes don't cause failures due to overloading the PSU, as such. Yes, a spike to 600W from a 350W GPU sounds bad, but the spikes are spikes -- they only happen for a fraction of a second. The wattage rating of the PSU is supposed to be for continuous load.

For spikes, what matters is how large the capacitors in the PSU are. And yes, bigger PSUs come with bigger capacitors, but nothing is stopping you from putting a set of large support capacitors in a not-as-huge PSU, people just tend to not. That's all in the general category of 'PSU quality', which is unfortunately hard to review.

If your system spikes to 900W, then sure, a 1000W PSU will handle the load. It's just overkill. ... if it were possible to easily find a lower-rating PSU that can handle those spikes.