AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
138.62k stars 26.32k forks source link

[Bug]: crash when using sdxl loras #15175

Open TongfanWeitf opened 5 months ago

TongfanWeitf commented 5 months ago

Checklist

What happened?

if i use sdxl loras, webui will crash.

Steps to reproduce the problem

1.run webui. 2.run a txt2img with sdxl model and lora 3.crash

What should have happened?

successfully return the img

What browsers do you use to access the UI ?

Microsoft Edge

Sysinfo

sysinfo-2024-03-07-18-39.json

Console logs

Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.8.0
Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5
Launching Web UI with arguments: --xformers --no-half-vae --no-half --medvram-sdxl
Loading weights [67ab2fd8ec] from D:\ai\webui\models\Stable-diffusion\ponyDiffusionV6XL_v6StartWithThisOne.safetensors
Creating model from config: D:\ai\webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 16.4s (prepare environment: 3.4s, import torch: 5.9s, import gradio: 0.6s, setup paths: 0.8s, initialize shared: 3.2s, other imports: 0.6s, load scripts: 0.8s, create ui: 0.5s, gradio launch: 0.6s).
Loading VAE weights specified in settings: D:\ai\webui\models\VAE\sdxl_vae.safetensors
Applying attention optimization: xformers... done.
Model loaded in 20.6s (load weights from disk: 0.7s, create model: 1.9s, apply weights to model: 7.4s, apply float(): 4.6s, load VAE: 0.7s, calculate empty prompt: 5.3s).
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:25<00:00,  1.28s/it]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:27<00:00,  1.40s/it]
  0%|                                                                                           | 0/20 [00:00<?, ?it/s] 请按任意键继续. . .
//the first bar is without lora and the second one is with lora. it crashed so no error messages. the chinese at the end means "press any key to continue..."

Additional information

it is weird, beacuse I can run sdxl with loras before. In some day I suddently cant load sdxl models (pytorch allocated 10.6G which is much more than before), so I add --medvram-sdxl. Now I can load sdxl models, but I still cant use loras.

TongfanWeitf commented 5 months ago

i uploaded my gpu drive, now it reaches the end of progress: Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] Version: v1.8.0 Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5 Launching Web UI with arguments: --xformers --no-half-vae --no-half --medvram-sdxl Loading weights [67ab2fd8ec] from D:\ai\webui\models\Stable-diffusion\ponyDiffusionV6XL_v6StartWithThisOne.safetensors Creating model from config: D:\ai\webui\repositories\generative-models\configs\inference\sd_xl_base.yaml Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). Startup time: 13.2s (prepare environment: 3.1s, import torch: 6.0s, import gradio: 0.6s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.5s, load scripts: 0.8s, create ui: 0.5s, gradio launch: 0.5s). Applying attention optimization: xformers... done. Model loaded in 16.6s (load weights from disk: 0.7s, create model: 2.2s, apply weights to model: 7.4s, apply float(): 4.9s, calculate empty prompt: 1.3s). 100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:51<00:00, 2.57s/it] Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:33<00:00, 1.75s/it]

but will still crash after that(means I cant receive the img, though i can see it is gerenated, but it crashed the moment it finished, the eta in webui is like 90%/3s left)

nailz420 commented 5 months ago

Can you check in windows event logs for any related messages? Python crash details? Resource exhaustion?

TongfanWeitf commented 5 months ago

Can you check in windows event logs for any related messages? Python crash details? Resource exhaustion?

i already solved it. when I want to load sdxl models, I use --medvram-sdxl, then I restart without --medvram-sdxl and then I can used sdxl models. If I want to load another model, I restart with --medvram-sdxl again and do such thing again.

nailz420 commented 5 months ago

Can you check in windows event logs for any related messages? Python crash details? Resource exhaustion?

i already solved it. when I want to load sdxl models, I use --medvram-sdxl, then I restart without --medvram-sdxl and then I can used sdxl models. If I want to load another model, I restart with --medvram-sdxl again and do such thing again.

That doesn't sound like a solution, but a clunky workaround. The issue still exists if you have to use that

aurelion314 commented 5 months ago

I have the same issue, but the solution of using --medvram-sdxl then restarting does not solve it for me.

The sdxl model works fine on its own, but as soon as I add the lora it crashes. Typically as soon as I try to generate, my computer temporarily locks up, then the ui crashes with this in the console:

0%| | 0/8 [00:00<?, ?it/s]./webui.sh: line 292: 11929 Killed "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"

Running Linux Mint. Using --no-half --precision full

Edit: I have discovered that when generating with a lora, it consumes all available RAM and then crashes. I suspect it is loading the entire base model into memory again, creating a copy and doubling memory usage. When only the base model is loaded, RAM usage is less than 20gb of 32gb. When I start the generation with the lora, roughly 3gb is added every seconds until it hits the full 32gb, which is when it either crashes or locks the PC.

I confirmed this by using SD 1.5, which is small enough that I can have 2 full copies in memory. Generating with a lora again starts by consuming a bunch of RAM, but stops at about 6gb of additional memory (the lora itself is only 150mb), then everything works and the image generates just fine.

I'm guessing that isn't supposed to happen? Shouldn't it either use the model already in memory, or free up that space if it is going to reload the whole thing?

aearone commented 5 months ago

I have the same issue, but the solution of using --medvram-sdxl then restarting does not solve it for me.

The sdxl model works fine on its own, but as soon as I add the lora it crashes. Typically as soon as I try to generate, my computer temporarily locks up, then the ui crashes with this in the console:

0%| | 0/8 [00:00<?, ?it/s]./webui.sh: line 292: 11929 Killed "${python_cmd}" -u "${LAUNCH_SCRIPT}" "$@"

Running Linux Mint. Using --no-half --precision full

Edit: I have discovered that when generating with a lora, it consumes all available RAM and then crashes. I suspect it is loading the entire base model into memory again, creating a copy and doubling memory usage. When only the base model is loaded, RAM usage is less than 20gb of 32gb. When I start the generation with the lora, roughly 3gb is added every seconds until it hits the full 32gb, which is when it either crashes or locks the PC.

I confirmed this by using SD 1.5, which is small enough that I can have 2 full copies in memory. Generating with a lora again starts by consuming a bunch of RAM, but stops at about 6gb of additional memory (the lora itself is only 150mb), then everything works and the image generates just fine.

I'm guessing that isn't supposed to happen? Shouldn't it either use the model already in memory, or free up that space if it is going to reload the whole thing?

I can confirm that when using the SDXL model (in my case PonyDiffusion), the RAM consumption increases with every second of generation until it reaches 15.6 GB (I have 16 GB) and the swap file starts to be used. That said, if I use Lora, the RAM gets clogged up after the first generation and it says "Press any button...". I perfectly remember using SDXL model 2 months ago without any problems, I don't remember such RAM consumption. And the error is not on the video memory side of the GPU. Has anyone found a solution?

aearone commented 5 months ago

I figured out what the problem is. The issue is a RAM leak in the latest version of webui, i.e. during generation the RAM usage grows every second until it reaches the swap file limit. In webui version 1.7.0 there was no such thing. It's just that RAM is at its maximum, it is enough for the first/second generation. But when using Lora, it runs out almost instantly. Apparently there is a bug somewhere that the model is unloaded into RAM at every generation. After downgrading the webui version, the memory leaks no longer occur and the images are generated normally:) As I understand it, this is because of PyTorch 2.1.2, which was added to webui 1.8.0 version.

Let's wait for fixes, if there are any in the future.

You can download version 1.7.0 manually here:: https://github.com/AUTOMATIC1111/stable-diffusion-webui/tree/v1.7.0

@https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/500#issue-2171364987 @https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/15206#issue-2177705949

Les-Tin commented 5 months ago

https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/15348

Stellaris12 commented 4 months ago

I figured out what the problem is. The issue is a RAM leak in the latest version of webui, i.e. during generation the RAM usage grows every second until it reaches the swap file limit. In webui version 1.7.0 there was no such thing. It's just that RAM is at its maximum, it is enough for the first/second generation. But when using Lora, it runs out almost instantly. Apparently there is a bug somewhere that the model is unloaded into RAM at every generation. After downgrading the webui version, the memory leaks no longer occur and the images are generated normally:) As I understand it, this is because of PyTorch 2.1.2, which was added to webui 1.8.0 version.

Let's wait for fixes, if there are any in the future.

You can download version 1.7.0 manually here:: https://github.com/AUTOMATIC1111/stable-diffusion-webui/tree/v1.7.0

@lllyasviel/stable-diffusion-webui-forge#500 (comment) @#15206 (comment)

I don't think torch 2.1.2 is the only cause of the issue as I am having this same issue with my ARC gpu which uses torch: 2.0.0a0+gite9ebda2 in version 1.8 of the webui.

TongfanWeitf commented 4 months ago

Can you check in windows event logs for any related messages? Python crash details? Resource exhaustion?

i already solved it. when I want to load sdxl models, I use --medvram-sdxl, then I restart without --medvram-sdxl and then I can used sdxl models. If I want to load another model, I restart with --medvram-sdxl again and do such thing again.

That doesn't sound like a solution, but a clunky workaround. The issue still exists if you have to use that

sry I cant give any debug logs. I am using google cloud and thus I can get rid of this bug. Just by restart the vm.

bandifiu commented 4 months ago

I had similar problem with RAM being filled after each generation when I tried to compare models with x/y/z plot. With v1.8 increasing the number of model loaded into RAM crashing the system after few generations. Memory is allocated after each generation even when model is used from cache. For me using the old settings (obsolate one) for increasing the number of cached models instead the new one fixed the crashes.

settings

silveroxides commented 4 months ago

This might be similar to my issues so I will post here instead of making new issue.

Console Log

Events look like this

Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: python.exe (52724) consumed 21200195584 bytes, msedge.exe (46448) consumed 6455615488 bytes, and python.exe (10724) consumed 6229950464 bytes.

Bytes converted. It shouold not consume that much right? hc4i87vPW1

Event Viewer 1

Event Viewer 2

silveroxides commented 4 months ago

@w-e-w You might want to look at this Update: I can somewhat see what might be happening now. Several times now when decoding image with VAE it leaves shared memory occupied while it drops some weights from the loaded checkpoint it seems which it then has to load back up when generation starts.

  1. End of normal run with decode start
  2. Decode where shared memory is used
  3. Decode finishes
  4. Normal loaded model weights
  5. Changing model

l5LBj9R7YK

  1. End of normal run with decode start
  2. Decode where shared memory is used
  3. Decode finishes and something is left in shared while VRAM is below normal
  4. Initiating next generation
  5. Actually starting to generate image

LzlhJyOkto

  1. Normal generation ongoing
  2. Decode start
  3. Decore using shared memory
  4. Same as previous with odd memory usage
  5. Sketchy workaround with opening settings and unloading model to RAM and then back to VRAM to get it back to normal

K01KjQmr5t

God-damnit-all commented 4 months ago

I notice that the 1.9 release candidate made some changes regarding loras, has anyone tested to see if it's fixed on there?

Takezo1000 commented 4 months ago

I tested SDXL loras with 1.9 and RAM usage still skyrockets to the maximum (32 GB) and then it starts using virtual memory (using up to 20 GB of virtual memory). Only happens when using SDXL Loras.

God-damnit-all commented 4 months ago

I tested SDXL loras with 1.9 and RAM usage still skyrockets to the maximum (32 GB) and then it starts using virtual memory (using up to 20 GB of virtual memory). Only happens when using SDXL Loras.

Is this with the now-released 1.9?

Takezo1000 commented 4 months ago

I tested SDXL loras with 1.9 and RAM usage still skyrockets to the maximum (32 GB) and then it starts using virtual memory (using up to 20 GB of virtual memory). Only happens when using SDXL Loras.

Is this with the now-released 1.9?

Yes, with 1.9 it still has this problem. I'm using --medvram --medvram-sdxl arguments with RX 6700 XT (AMD)

God-damnit-all commented 4 months ago

I tested SDXL loras with 1.9 and RAM usage still skyrockets to the maximum (32 GB) and then it starts using virtual memory (using up to 20 GB of virtual memory). Only happens when using SDXL Loras.

Is this with the now-released 1.9?

Yes, with 1.9 it still has this problem. I'm using --medvram --medvram-sdxl arguments with RX 6700 XT (AMD)

That's unfortunate. I'd downgrade to 1.7 but that has problems of its own. Could it be related to xformers? Try setting Cross attention optimization to None in Optimizations.

silveroxides commented 4 months ago

I tested SDXL loras with 1.9 and RAM usage still skyrockets to the maximum (32 GB) and then it starts using virtual memory (using up to 20 GB of virtual memory). Only happens when using SDXL Loras.

Is this with the now-released 1.9?

Yes, with 1.9 it still has this problem. I'm using --medvram --medvram-sdxl arguments with RX 6700 XT (AMD)

That's unfortunate. I'd downgrade to 1.7 but that has problems of its own. Could it be related to xformers? Try setting Cross attention optimization to None in Optimizations.

The following is what I did and it worked wonders. I get the feeling that the last two releases implemented a lot of arbritrary changes which had no justifiable reason to be implemented to a main branch. This include spandrel which has cause major slowdowns for a big part of users(yes you might not see them report here cause non tech savvy users get put off being requested to spend an hour learning how to properly submit an issue). I suspect they missed reading the full changelog for Torch releases 2.1.0-2.1.2 in regards to weights and multiple devices (in this case their odd loading between cpu and gpu) Anyway I hope this helps

Instructions:

Downgrade to 1.8.0

Install CUDA toolkit 11.8

Install cudnn

"extract respective files into CUDA installs bin,include and lib\x64 folders"

Click windows search bar

and type environment then click "Edit system environment variables" in results and in next window that pops up.

Editing variables

In the lower part check that you have CUDA_PATH set to same as CUDA_PATH_V11_8 and add one named CUDNN and give it the variable(assuming you install instandard location. edit the following if different)

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\lib\x64;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include;

After all this is done

Reboot your PC once.

Open your webui root folder

Open Powershell and acitvate the venv

.\venv\Scrips\activate

Then install torch 2.0.1 for cuda11.8

pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118

Install xformers 0.0.22

pip install xformers==0.0.22

Lastly do check just in case there is something important that conflicts but there rarely is.

pip check

Open webui-user.bat in text editor and add --skip-install just in case.

Hope this works. Will try to check back later to see if all went well

JeffreyLebino commented 4 months ago

Hi, happened to me this weekend as well,

Tried rolling back the 1.9.3 back to 1.9 then 1.8 as well as deleting the venv before coming here but it seems it didnt work. Anytime i try to gen on 1.5 the ram goes up then returns to about a fourth of the total available. With SDXL it seems there is a memory leak and it climbs with each gen, sometimes lowering a little but eventually filling the total.

Tried on 16 and 32gb of ram and i get the same effect on both. Adding more loras make the server crash earlier. I tried switching to 1.5 models after each XL gens but it seems to be mostly placebo. tried switching off xformers but no results either.

It pretty much always ends with an error message or with just "press any key to continue..." cmd_EjESY1zMy4 firefox_JPd8HggDnf

aearone commented 4 months ago

Hi, happened to me this weekend as well,

Tried rolling back the 1.9.3 back to 1.9 then 1.8 as well as deleting the venv before coming here but it seems it didnt work. Anytime i try to gen on 1.5 the ram goes up then returns to about a fourth of the total available. With SDXL it seems there is a memory leak and it climbs with each gen, sometimes lowering a little but eventually filling the total.

Tried on 16 and 32gb of ram and i get the same effect on both. Adding more loras make the server crash earlier. I tried switching to 1.5 models after each XL gens but it seems to be mostly placebo. tried switching off xformers but no results either.

It pretty much always ends with an error message or with just "press any key to continue..." cmd_EjESY1zMy4 firefox_JPd8HggDnf

I can attest to that. The same thing is happening to me. Same problem on version 1.8.0, the failures are less but after a few generations there is still a RAM error. I found out that the problem lies in the command --medvram, it frees the video memory and very heavily loads the RAM, while the VRAM barely uses 6.5 gb (out of 8 in my case). If you remove this command, the generation takes a very long time. I don't know what's broken in the latest webui updates, but it's a fact. There is a leak of RAM when it is used at 100%, and if you use LoRA it happens even faster.

nailz420 commented 4 months ago

Noticed that this memory leak is not triggered during X/Y/Z script runs

JeffreyLebino commented 3 months ago

Ok i may have a workaround that limits the amount of crashes, while using Kohya i had the same issue an it resolved both, mind you it still fills your ram to an absurd amount (31.9/32 for me) it seems to just avoid the final crash when it tries to add more to it

Disabling system memory fallback helps by stopping the gpu from dumping the excess of VRAM into RAM (it might even speed up your 1.5 gens if you havent done it in the past) Theres still something filling 99.9% of the system RAM, and SD still redlines all the time but at least the gpu doesnt dump more into that 0.1%

https://nvidia.custhelp.com/app/answers/detail/a_id/5490/~/system-memory-fallback-for-stable-diffusion Just be sure to do it on the right python, the venv or the main system one depending on which one your SD uses.

nailz420 commented 3 months ago

Noticed that this memory leak is not triggered during X/Y/Z script runs

Running X/Y/Z plot script actually clears out the VM

nailz420 commented 3 months ago

Ok i may have a workaround that limits the amount of crashes, while using Kohya i had the same issue an it resolved both, mind you it still fills your ram to an absurd amount (31.9/32 for me) it seems to just avoid the final crash when it tries to add more to it

Disabling system memory fallback helps by stopping the gpu from dumping the excess of VRAM into RAM (it might even speed up your 1.5 gens if you havent done it in the past) Theres still something filling 99.9% of the system RAM, and SD still redlines all the time but at least the gpu doesnt dump more into that 0.1%

https://nvidia.custhelp.com/app/answers/detail/a_id/5490/~/system-memory-fallback-for-stable-diffusion Just be sure to do it on the right python, the venv or the main system one depending on which one your SD uses.

You're supposed to do that for SD regardless of this bug, especially with 8gb VRAM or less

JeffreyLebino commented 3 months ago

You're supposed to do that for SD regardless of this bug, especially with 8gb VRAM or less

Sure but OP and i both had crashes as well as the memory leak hence why i shared it, two problems in one sort of things. I experienced way less issues with the fix. And SDXL was working fine without it on my side so far.

AlanMW commented 3 months ago

My webUI stopped hanging up when I removed the --medvram flag as well. It cuts my speed from 6.5 it/s to 3 it/s but at least it runs.

nailz420 commented 3 months ago

I use Intelligent standby list cleaner (ISLC). 8gb VRAM, 32gb RAM. I can generate for about 20-30 mins before crashing with medvram-sdxl and xformers.

aearone commented 3 months ago

I use Intelligent standby list cleaner (ISLC). 8gb VRAM, 32gb RAM. I can generate for about 20-30 mins before crashing with medvram-sdxl and xformers.

So it's a massive problem. We should do something about it and draw the contributors attention to it.

PKK02 commented 3 months ago

I've been facing the same exact problem for weeks now. I was able to use XL loras before with no problems and now it crashes my entire system even if I use it only once. It will go through the first gen but if I try to generate other images, it slowly freezes my computer even when not using loras. Very annoying. I'm using a GTX1080 on Arch Linux. Latest drivers, latest webui, --medvram-sdxl flag.

Euchale commented 2 months ago

Just adding myself in as well with this problem. Same boat, was able to use it fine, but can't use it anymore.

PKK02 commented 2 months ago

In case people are still having this problem on Auto, I managed to use loras with Forge (https://github.com/lllyasviel/stable-diffusion-webui-forge). It's basically auto but much faster. It's apparently been abandoned but for what I do it works just fine. I can use multiple loras at the same time with no problems. There's an asterisk however. If I try to swap >checkpoints< in a session it still freezes my system so there's that.

aurelion314 commented 2 months ago

In case people are still having this problem on Auto, I managed to use loras with Forge (https://github.com/lllyasviel/stable-diffusion-webui-forge).

Thanks for the suggestion! I've been watching this thread hoping for a fix, but until then it's nice to have another option.

nekoworkshop commented 2 months ago

Bumping this since I'm currently on dev branch (v1.9.4-169-ga30b19dd) and am seemingly experiencing similar issues.

I have 48GB of RAM so I didn't notice the growing leak until my C drive running out of space due to the page file growing to 60GBs. That's a nearly a 100GB commit limit. image

Here are the numbers from VMMap.

image

When idling, the Working Set size is about 7GB. But the program holds a huge amount of memory committed. image

God-damnit-all commented 2 months ago

This is getting buried when it's a severe issue. Desperation time. I apologize in advance: @w-e-w @AUTOMATIC1111 @akx @catboxanon @KohakuBlueleaf @missionfloyd @light-and-ray

wfjsw commented 2 months ago

Please set environment variable PYTHONFAULTHANDLER=1 or use -X faulthandler and then trigger the error again. Upload the full log after crash, together with the dump file located in %LOCALAPPDATA%\CrashDumps.

nailz420 commented 1 month ago

crash.txt crash2.txt

Tell me what commands to execute on the crash dump. I don't want to share it.

Please set environment variable PYTHONFAULTHANDLER=1 or use -X faulthandler and then trigger the error again. Upload the full log after crash, together with the dump file located in %LOCALAPPDATA%\CrashDumps.

wfjsw commented 1 month ago

There is a high chance that I need more than it. Let's wait until someone shares.

nailz420 commented 1 month ago

There is a high chance that I need more than it. Let's wait until someone shares.

Is there any way I can share it privately?

wfjsw commented 1 month ago

There is a discord channel on the wiki, if you want to join.

nailz420 commented 1 month ago

There is a discord channel on the wiki, if you want to join.

I don't see any discord mention on A1111 wiki. However, I am on the stable diffusion discord. I don't see your (wfjsw) nickname there though.

wfjsw commented 1 month ago

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Contributing

It is hidden deep for some reason.

aearone commented 1 month ago

After the latest webui update I can confirm that the memory leak is fixed, the system does not consume more than 20-25gb of RAM when generating using LoRA and does not crash with an error.

nailz420 commented 1 month ago

After the latest webui update

What latest update? What branch?

aearone commented 1 month ago

After the latest webui update

What latest update? What branch?

https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/9e404c315432ca9aca2b9805e462c2360b19f5ae

nailz420 commented 1 month ago

modules/models/sd3/sd3_model.py

Isn't this only for SD3?

On Wed, Jul 3, 2024 at 8:16 AM Nikolay L. @.***> wrote:

After the latest webui update

What latest update? What branch?

9e404c3 https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/9e404c315432ca9aca2b9805e462c2360b19f5ae

— Reply to this email directly, view it on GitHub https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/15175#issuecomment-2205939211, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGBCKPHCBJVHDS5QMD3IA6LZKPTSPAVCNFSM6AAAAABELRFZX6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBVHEZTSMRRGE . You are receiving this because you commented.Message ID: @.***>

aearone commented 1 month ago

modules/models/sd3/sd3_model.py Isn't this only for SD3? On Wed, Jul 3, 2024 at 8:16 AM Nikolay L. @.> wrote: After the latest webui update What latest update? What branch? 9e404c3 <9e404c3> — Reply to this email directly, view it on GitHub <#15175 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGBCKPHCBJVHDS5QMD3IA6LZKPTSPAVCNFSM6AAAAABELRFZX6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBVHEZTSMRRGE . You are receiving this because you commented.Message ID: @.>

This update also extended to the main webui. At least my branch was updated when I started webui and the memory leak stopped when I used --medvram

nailz420 commented 1 month ago

There haven't been any commits to the master branch since May 28, 2024. Last update on a branch was 4 days ago.

aearone commented 1 month ago

There haven't been any commits to the master branch since May 28, 2024. Last update on a branch was 4 days ago.

Yes, that's correct, and yet my webui got the updates when running with the git pull command.

BlackWyvern commented 1 month ago

git checkout'd back to master and did a git pull. I can confirm that SDXL models no longer use 24+GB of my ram whilst idle and try to crash my system on execute. And SD1.5 uses a 'mere' 10GB compared to the 18 it kept trying to allocate to.

But I also now am completely unable to switch models due to new cuda memory errors.. And SDXL fails to complete due to "Press any key to continue..." at 100% which event viewer points to an error in c10.dll. https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/15430