search

huggingface / diffusers

๐Ÿค— Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
26.14k stars 5.38k forks source link

RuntimeWarning: invalid value encountered in cast images = (images * 255).round().astype("uint8") output black image #6815

Closed pavankay closed 8 months ago

pavankay commented 9 months ago

Describe the bug

When running the stable-diffusion-2-1 I get a runtime warning "RuntimeWarning: invalid value encountered in cast images = (images * 255).round().astype("uint8")" and the image output is black. astronaut_rides_horse

Reproduction

My code: import torch from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler

model_id = "stabilityai/stable-diffusion-2-1"

comment#Use the DPMSolverMultistepScheduler (DPM-Solver++) scheduler here instead pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16) pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) pipe = pipe.to("cuda")

prompt = "a photo of an astronaut riding a horse on mars" image = pipe(prompt).images[0]

image.save("astronaut_rides_horse.png")

I got it from here: https://huggingface.co/stabilityai/stable-diffusion-2-1

Logs

Output:
PS C:\AI\diffusion> & c:/AI/diffusion/.conda/python.exe c:/AI/diffusion/main.py
model_index.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 537/537 [00:00<00:00, 556kB/s]
C:\AI\diffusion\.conda\lib\site-packages\huggingface_hub\file_download.py:149: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\user\.cache\huggingface\hub\models--stabilityai--stable-diffusion-2-1. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)
(โ€ฆ)ature_extractor/preprocessor_config.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 342/342 [00:00<?, ?B/s]
tokenizer/tokenizer_config.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 824/824 [00:00<?, ?B/s]
tokenizer/special_tokens_map.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 460/460 [00:00<?, ?B/s] 
tokenizer/merges.txt: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 525k/525k [00:00<00:00, 29.9MB/s] 
scheduler/scheduler_config.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 345/345 [00:00<?, ?B/s] 
tokenizer/vocab.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 1.06M/1.06M [00:00<00:00, 23.0MB/s] 
text_encoder/config.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 633/633 [00:00<?, ?B/s] 
vae/config.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 611/611 [00:00<?, ?B/s] 
unet/config.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 939/939 [00:00<?, ?B/s] 
diffusion_pytorch_model.safetensors: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 335M/335M [00:11<00:00, 29.8MB/s] 
model.safetensors: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 1.36G/1.36G [00:31<00:00, 42.7MB/s] 
diffusion_pytorch_model.safetensors: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 3.46G/3.46G [01:17<00:00, 44.7MB/s] 
Fetching 13 files: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 13/13 [01:18<00:00,  6.07s/it] 
Loading pipeline components...: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 6/6 [00:02<00:00,  2.71it/s] 
100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 50/50 [04:57<00:00,  5.95s/it]
C:\AI\diffusion\.conda\lib\site-packages\diffusers\image_processor.py:90: RuntimeWarning: invalid value encountered in cast
  images = (images * 255).round().astype("uint8")
PS C:\AI\diffusion> ^C
PS C:\AI\diffusion>

System Info

Who can help?

No response

Bhavay-2001 commented 9 months ago

Hi @sayakpaul @yiyixuxu, I would like to work on this issue. Please let me know how can I start. Thanks

CTimmerman commented 9 months ago

Same on my GTX 1660 Ti with CUDA 12.3.

Replace torch.float16 with torch.float32 to fix.

Same for webui using: webui.bat --no-half

Before: ``` D:\code\AI\Python>nvidia-smi Sun Feb 4 16:18:35 2024 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 546.12 Driver Version: 546.12 CUDA Version: 12.3 | |-----------------------------------------+----------------------+----------------------+ | GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:01:00.0 On | N/A | | N/A 50C P8 7W / 80W | 1371MiB / 6144MiB | 7% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 9940 C+G ...r and Support Assistant\DSATray.exe N/A | | 0 N/A N/A 33880 C+G ...werToys\PowerToys.PowerLauncher.exe N/A | | 0 N/A N/A 90960 C ...rograms\Python\Python310\python.exe N/A | | 0 N/A N/A 109548 C+G ...werToys\PowerToys.ColorPickerUI.exe N/A | | 0 N/A N/A 128552 C+G ...PowerToys\PowerToys.PowerAccent.exe N/A | +---------------------------------------------------------------------------------------+ d:\code\AI\Python>python du_lora.py C:\Users\C\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. torch.utils._pytree._register_pytree_node( Loading pipeline components...: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 7/7 [00:11<00:00, 1.68s/it] The config attributes {'skip_prk_steps': True} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file. 0%| | 0/8 [00:00python du_lora.py C:\Users\C\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. torch.utils._pytree._register_pytree_node( Loading pipeline components...: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 7/7 [00:06<00:00, 1.09it/s] The config attributes {'skip_prk_steps': True} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file. 0%| | 0/8 [00:00python du_lora.py # C:\Users\C\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. # torch.utils._pytree._register_pytree_node( # Loading pipeline components...: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 7/7 [00:11<00:00, 1.68s/it] # The config attributes {'skip_prk_steps': True} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file. # 0%| | 0/8 [00:00
Bhavay-2001 commented 9 months ago

Hi @CTimmerman, does this problem exists for torch.float16 on a windows system? Should the solution be crafted? Thanks

CTimmerman commented 9 months ago

@Bhavay-2001 Yes: Edition Windows 11 Home Version 23H2 Installed on โ€Ž28-โ€Ž11-โ€Ž2022 OS build 22631.3007 Experience Windows Feature Experience Pack 1000.22681.1000.0

Bhavay-2001 commented 9 months ago

I think we need to discuss this with @sayakpaul @yiyixuxu and then I can contribute in this.

CTimmerman commented 9 months ago

Enabling autocast is better than raising the error to the user and outputting black images, but i think that suddenly stopped working in my webgui until i started using --no-half there.

Bhavay-2001 commented 9 months ago

Soo how can we make this change? Any helps will be appreciated.

sayakpaul commented 9 months ago

We have made a number of fixes to the DPM schedulers. Could you try installing the latest version of diffusers and check again?

Cc: @yiyixuxu

CTimmerman commented 9 months ago

We have made a number of fixes to the DPM schedulers. Could you try installing the latest version of diffusers and check again?

Main doesn't look any different, and there are too many other branches to read. If this is not a problem limited to 16xx cards, then you can test it using my code by changing float32 back to float16.

I'm already using the main branch:

``` D:\code\AI\Python>python -m pip freeze absl-py==2.1.0 accelerate==0.26.1 aiohttp==3.9.3 aiosignal==1.3.1 async-timeout==4.0.3 attrs==23.2.0 cachetools==5.3.2 certifi==2023.11.17 charset-normalizer==3.3.2 colorama==0.4.6 datasets==2.16.1 defusedxml==0.7.1 diffusers @ file:///D:/code/AI/Python/diffusers dill==0.3.7 filelock==3.13.1 frozenlist==1.4.1 fsspec==2023.10.0 ftfy==6.1.3 google-auth==2.27.0 google-auth-oauthlib==1.2.0 grpcio==1.60.1 huggingface-hub==0.20.3 idna==3.6 importlib-metadata==7.0.1 Jinja2==3.1.3 keyboard==0.13.5 Markdown==3.5.2 MarkupSafe==2.1.4 mouse==0.7.1 mpmath==1.3.0 multidict==6.0.5 multiprocess==0.70.15 networkx==3.2.1 numpy==1.26.3 oauthlib==3.2.2 packaging==23.2 pandas==2.2.0 peft==0.7.0 pillow==10.2.0 protobuf==4.23.4 psutil==5.9.8 pyarrow==15.0.0 pyarrow-hotfix==0.6 pyasn1==0.5.1 pyasn1-modules==0.3.0 python-dateutil==2.8.2 pytz==2024.1 PyYAML==6.0.1 regex==2023.12.25 requests==2.31.0 requests-oauthlib==1.3.1 rsa==4.9 safetensors==0.4.2 six==1.16.0 sympy==1.12 tensorboard==2.15.1 tensorboard-data-server==0.7.2 tokenizers==0.15.1 torch==2.2.0+cu121 torchaudio==2.2.0+cu121 torchvision==0.17.0 tqdm==4.66.1 transformers==4.37.2 typing_extensions==4.9.0 tzdata==2023.4 urllib3==2.1.0 wcwidth==0.2.13 Werkzeug==3.0.1 xxhash==3.4.1 yarl==1.9.4 zipp==3.17.0 D:\code\AI\Python>python -m pip show diffusers Name: diffusers Version: 0.26.0.dev0 Summary: State-of-the-art diffusion in PyTorch and JAX. Home-page: https://github.com/huggingface/diffusers Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/diffusers/graphs/contributors) Author-email: patrick@huggingface.co License: Apache 2.0 License Location: c:\users\c\appdata\local\programs\python\python310\lib\site-packages Requires: filelock, huggingface-hub, importlib-metadata, numpy, Pillow, regex, requests, safetensors Required-by: D:\code\AI\Python>python -m pip install diffusers --upgrade Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Requirement already satisfied: diffusers in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (0.26.0.dev0) Collecting diffusers Downloading diffusers-0.26.1-py3-none-any.whl.metadata (18 kB) Requirement already satisfied: importlib-metadata in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from diffusers) (7.0.1) Requirement already satisfied: filelock in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from diffusers) (3.13.1) Requirement already satisfied: huggingface-hub>=0.20.2 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from diffusers) (0.20.3) Requirement already satisfied: numpy in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from diffusers) (1.26.3) Requirement already satisfied: regex!=2019.12.17 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from diffusers) (2023.12.25) Requirement already satisfied: requests in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from diffusers) (2.31.0) Requirement already satisfied: safetensors>=0.3.1 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from diffusers) (0.4.2) Requirement already satisfied: Pillow in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from diffusers) (10.2.0) Requirement already satisfied: fsspec>=2023.5.0 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from huggingface-hub>=0.20.2->diffusers) (2023.10.0) Requirement already satisfied: tqdm>=4.42.1 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from huggingface-hub>=0.20.2->diffusers) (4.66.1) Requirement already satisfied: pyyaml>=5.1 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from huggingface-hub>=0.20.2->diffusers) (6.0.1) Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from huggingface-hub>=0.20.2->diffusers) (4.9.0) Requirement already satisfied: packaging>=20.9 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from huggingface-hub>=0.20.2->diffusers) (23.2) Requirement already satisfied: zipp>=0.5 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from importlib-metadata->diffusers) (3.17.0) Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from requests->diffusers) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from requests->diffusers) (3.6) Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from requests->diffusers) (2.1.0) Requirement already satisfied: certifi>=2017.4.17 in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from requests->diffusers) (2023.11.17) Requirement already satisfied: colorama in c:\users\c\appdata\local\programs\python\python310\lib\site-packages (from tqdm>=4.42.1->huggingface-hub>=0.20.2->diffusers) (0.4.6) Downloading diffusers-0.26.1-py3-none-any.whl (1.9 MB) โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” 1.9/1.9 MB 20.5 MB/s eta 0:00:00 Installing collected packages: diffusers Attempting uninstall: diffusers Found existing installation: diffusers 0.26.0.dev0 Uninstalling diffusers-0.26.0.dev0: Successfully uninstalled diffusers-0.26.0.dev0 Successfully installed diffusers-0.26.1 D:\code\AI\Python>cd diffusers && git status On branch main Your branch is behind 'origin/main' by 1 commit, and can be fast-forwarded. (use "git pull" to update your local branch) nothing to commit, working tree clean D:\code\AI\Python\diffusers>git pull Updating 13001ee3..fbdf26ba Fast-forward examples/dreambooth/train_dreambooth_lora_sdxl.py | 100 +++++++++++++++------- 1 file changed, 67 insertions(+), 33 deletions(-) ``` That code is not used in this issue.
pavankay commented 9 months ago

how can I fix the problem though this has also been happening with the upscaler models too. When I run it in google colab it works.

github-actions[bot] commented 8 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

sayakpaul commented 8 months ago

Gently pinging @yiyixuxu.

yiyixuxu commented 8 months ago

I'm not sure what to do here other than use full precision It is a known issue with GTX 1660 https://github.com/huggingface/diffusers/issues/2153

jloveric commented 8 months ago

I think I'm seeing this same issue with on a 4090 on ubuntu, using the train_text_to_image_lora_sdxl script 

accelerate launch train_text_to_image_lora_sdxl.py   --mixed_precision=fp16 --pretrained_model_name_or_path=$MODEL_NAME   --train_data_dir=instance-imgs   --dataloader_num_workers=8  --output_dir=outputxl   --report_to=tensorboard   --checkpointing_steps=500   --validation_prompt="some prompts"   --seed=42 --train_batch_size=1 --learning_rate=1e-4

dropping lr to 1e-6 doesn't help. The images passed to tensorboard are black. Could be other default parameters are causing the issue. The same dataset works fine the the sd1.5 lora trainer.

sayakpaul commented 8 months ago

You might want to use a more numerically stable VAE: "madebyollin/sdxl-vae-fp16-fix" and pass it to pretrained_vae_model_name_or_path. I think we show this already from https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README_sdxl.md.

jloveric commented 8 months ago

Thanks @sayakpaul that solves the problem for me

Darkrred commented 5 months ago

ๆ‰ฉๆ•ฃๆจกๅž‹็š„้ข„่ฎญ็ปƒๆƒ้‡ๅœจๆŽจ็†ๆ—ถไผšๅ‘็”Ÿๅ˜ๅŒ–๏ผŒ้‡ๆ–ฐไธ‹่ฝฝ้ข„่ฎญ็ปƒๆƒ้‡ๅณๅฏ่งฃๅ†ณ้—ฎ้ข˜

exdysa commented 1 month ago

Just in case someone else chances upon this thread... \Lib\site-packages\diffusers\image_processor.py:111: RuntimeWarning: invalid value encountered in cast images = (images * 255).round().astype("uint8") Was able to avoid this by toggling rescale_betas_zero_snr while using DDIMScheduler