vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.58k stars 408 forks source link

[Issue]: dev 5e1da44b bad XYZ performance and errors #3374

Closed MysticDaedra closed 1 month ago

MysticDaedra commented 1 month ago

Issue Description

Latest dev (5e1da44b), terrible performance and errors appearing in console that were not there last night when doing a grid. Errors include:

XYZ grid: Failed to process image: Allocation on device

and

XYZ grid: OutOfMemoryError

and

XYZ grid: no processing results

I started an XYZ grid to test new DoRA epochs. Been testing DoRA training results pretty much every day the past week, never had this issue before tonight.

Version Platform Description

Python version 3.11.8 SD.Next Dev 5e1da44b Windows 11 Professional 23H2 build 22631.4037 RTX 3070 8GB 32GB RAM Ryzen 7 5700X @ 4.8GHz Mozilla Firefox 129.0.1 Torch 2.3.1+cu121, diffusers 0.30.0, gradio 3.43.2 CUDA 12.1, cudnn 8907 Extensions enabled: Lora, sd-extension-chainner, sd-extension-system-info, sd-webui-agent-scheduler, stable-diffusion-webui-images-browser, stable-diffusion-webui-rembg, adetailer, canvas-zoom, sd-webui-infinite-image-browsing, sd-webui-temporal, ultimate-upscale-for-automatic1111

Relevant log output

https://discord.com/channels/1101998836328697867/1130536562422186044/1274616807361351701

Backend

Diffusers

UI

Standard

Branch

Dev

Model

StableDiffusion XL

Acknowledgements

MysticDaedra commented 1 month ago

Apparently some of the settings for --medvram changed. I did not have the --medvram flag on my shortcut, I was relying on the setting, but this has apparently been removed. Re-enabling CPU offload, dynamic attention, and VAE tiling solved the performance issues.