search

lllyasviel / Fooocus

Focus on prompting and generating
GNU General Public License v3.0
41.19k stars 5.8k forks source link

[Bug]: CUDA out of memory with a Tesla K80 #2436

Closed giuseppelibrandi closed 7 months ago

giuseppelibrandi commented 8 months ago

Checklist

What happened?

I have a VPS witha tesla K80 / Eight-Core Xeon E5-2690 / 64GB RAM / 1080GB SSD and i still have the error

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 622.00 MiB. GPU 0 has a total capacity of 11.17 GiB of which 166.25 MiB is free. Including non-PyTorch memory, this process has 11.01 GiB memory in use. Of the allocated memory 10.36 GiB is allocated by PyTorch, and 215.58 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

Steps to reproduce the problem

just start Fooocus, installed on a venv following the guide

What should have happened?

It should generate the image without running out of memory

What browsers do you use to access Fooocus?

Brave, Apple Safari, iOS

Where are you running Fooocus?

Locally

What operating system are you using?

Linux Ubuntu

Console logs

(venv_Fooocus) administrator@yourname:~/venv_Fooocus/Fooocus$ python3 entry_with_update.py
Already up-to-date
Update succeeded.
[System ARGV] ['entry_with_update.py']
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Fooocus version: 2.2.0
Total VRAM 11441 MB, total RAM 64264 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 Tesla K80 : native
VAE dtype: torch.float32
Using pytorch cross attention
Refiner unloaded.
Running on local URL: http://127.0.0.1:7865
To create a public link, set `share=True` in `launch()`.
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
Base model loaded: /home/administrator/venv_Fooocus/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [/home/administrator/venv_Fooocus/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors].
Loaded LoRA [/home/administrator/venv_Fooocus/Fooocus/models/loras/sd_xl_offset_example-lora_1.0.safetensors] for UNet [/home/administrator/venv_Fooocus/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cpu, use_fp16 = False.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
Started worker with PID 1890
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ControlNet Softness = 0.25
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 8105991395323598190
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] a dog, intricate, elegant, highly detailed, wonderful colors, sweet, sharp focus, professional glowing, attractive, extremely detail, striking, cute, delicate, confident, colorful, beautiful, surreal, iconic, fine, inspirational, artistic, winning, perfect, epic, full color, light, atmosphere, cool, lovely, creative, positive, determined, radiant, loving
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] a dog, beautiful scenic full, perfect, intricate, elegant, highly detailed, wonderful colors,, shining, sharp focus, epic, cinematic, professional, stunning, attractive, artistic, best, dramatic, cute, creative, positive, joyful, vibrant, loving, pretty, determined, friendly, cool, relaxed, designed, inspiring, bright, inspired, lovely, futuristic
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 11.00 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 3.63 seconds
0%| | 0/30 [00:00<?, ?it/s]/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Applied workaround for CuDNN issue, install nvrtc.so (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:80.)
return F.conv2d(input, weight, bias, self.stride,
0%| | 0/30 [00:01<?, ?it/s]
Traceback (most recent call last):
File "/home/administrator/venv_Fooocus/Fooocus/modules/async_worker.py", line 896, in worker
handler(task)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/modules/async_worker.py", line 803, in handler
imgs = pipeline.process_diffusion(
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/modules/default_pipeline.py", line 362, in process_diffusion
sampled_latent = core.ksampler(
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/modules/core.py", line 308, in ksampler
samples = ldm_patched.modules.sample.sample(model,
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/modules/sample.py", line 100, in sample
samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/modules/samplers.py", line 712, in sample
return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/modules/sample_hijack.py", line 157, in sample_hacked
samples = sampler.sample(model_wrap, sigmas, extra_args, callback_wrap, noise, latent_image, denoise_mask, disable_pbar)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/modules/samplers.py", line 557, in sample
samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/k_diffusion/sampling.py", line 701, in sample_dpmpp_2m_sde_gpu
return sample_dpmpp_2m_sde(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, eta=eta, s_noise=s_noise, noise_sampler=noise_sampler, solver_type=solver_type)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/k_diffusion/sampling.py", line 613, in sample_dpmpp_2m_sde
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/modules/patch.py", line 321, in patched_KSamplerX0Inpaint_forward
out = self.inner_model(x, sigma,
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/modules/samplers.py", line 271, in forward
return self.apply_model(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/modules/samplers.py", line 268, in apply_model
out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed)
File "/home/administrator/venv_Fooocus/Fooocus/modules/patch.py", line 237, in patched_sampling_function
positive_x0, negative_x0 = calc_cond_uncond_batch(model, cond, uncond, x, timestep, model_options)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/modules/samplers.py", line 222, in calc_cond_uncond_batch
output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/modules/model_base.py", line 85, in apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/modules/patch.py", line 404, in patched_unet_forward
h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/ldm/modules/diffusionmodules/openaimodel.py", line 43, in forward_timestep_embed
x = layer(x, context, transformer_options)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/ldm/modules/attention.py", line 613, in forward
x = block(x, context=context[i], transformer_options=transformer_options)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/ldm/modules/attention.py", line 440, in forward
return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/ldm/modules/diffusionmodules/util.py", line 189, in checkpoint
return func(*inputs)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/ldm/modules/attention.py", line 500, in _forward
n = self.attn1(n, context=context_attn1, value=value_attn1)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/administrator/venv_Fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/ldm/modules/attention.py", line 392, in forward
out = optimized_attention(q, k, v, self.heads)
File "/home/administrator/venv_Fooocus/Fooocus/ldm_patched/ldm/modules/attention.py", line 325, in attention_pytorch
out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 622.00 MiB. GPU 0 has a total capacity of 11.17 GiB of which 166.25 MiB is free. Including non-PyTorch memory, this process has 11.01 GiB memory in use. Of the allocated memory 10.36 GiB is allocated by PyTorch, and 215.58 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Total time: 16.58 seconds

Additional information

No response

QuantumCrazy commented 8 months ago

Hi, I got similar issue. I restarted PC and issue was fixed. Memory was full. I suspect something is causing memory to be overloaded, in my case background processes continue running after Fooocus crashed in browser (I got error messages). Then I open a new instance and it cannot run. I had to manually kill Fooocus processes in system monitor.

I think something with the recent release (2.2.0) it crashes more often (previous version of Fooocus before 2.2.0 did not crash at all for me).

giuseppelibrandi commented 8 months ago

Unfortunately i've already tried restarting the VPS but i get the same error

mashb1t commented 8 months ago

Duplicate of https://github.com/lllyasviel/Fooocus/issues/1984

2.2.0 didn't really change a lot regarding memory consumption, so this may not be related. Running fine on my RTX 3080 with 10GB VRAM, 32GB RAM and swap enabled.

Did you set up swap so VRAM can be offloaded when more is needed? Can you check if you can run 2.1.865 by checking the tag out with it and run python3 launch.py? Thanks!

giuseppelibrandi commented 8 months ago

Yes i have 40GB swap enabled:

(venv_fooocus) administrator@yourname:~/Fooocus$ swapon --show
NAME      TYPE SIZE USED PRIO
/swapfile file  40G   0B   -2

How can i try to run 2.1.865? this is the python3 launch.py 

(venv_fooocus) administrator@giuseppelibrandi:~/Fooocus$ python launch.py
[System ARGV] ['launch.py']
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Fooocus version: 2.2.0
Total VRAM 11441 MB, total RAM 64264 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 Tesla K80 : native
VAE dtype: torch.float32
Using pytorch cross attention
Refiner unloaded.
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.text_projection'}
Base model loaded: /home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [/home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors].
Loaded LoRA [/home/administrator/Fooocus/models/loras/sd_xl_offset_example-lora_1.0.safetensors] for UNet [/home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cpu, use_fp16 = False.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
Started worker with PID 2032
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ControlNet Softness = 0.25
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 1822662924200805213
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] a dog, intricate, elegant, highly detailed, wonderful colors, glowing, sharp focus, symmetry, fine detail, clear quality, artistic, color, winning, perfect composition, innocent, novel, beautiful, attractive, cute, cinematic, stunning, gorgeous, creative, positive, awesome, pure, determined, amazing, full, very coherent, consistent, vibrant, loving, pretty
[Fooocus] Preparing Fooocus text #2 ...
[Prompt Expansion] a dog, intricate, elegant, highly detailed, sharp focus, candid, charming, cute, expressive, pretty, confident, bright color, inspired, vibrant, elaborate, epic, beautiful, cinematic, fine detail, full background, professional, stunning, attractive, brilliant, creative, pure, wonderful, passionate, amazing, cool, awesome, gorgeous, illuminated, colorful, very
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding positive #2 ...
[Fooocus] Encoding negative #1 ...
[Fooocus] Encoding negative #2 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 11.07 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 2.78 seconds
  0%|                                                                                                             | 0/30 [00:00<?, ?it/s]/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Applied workaround for CuDNN issue, install nvrtc.so (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:80.)
  return F.conv2d(input, weight, bias, self.stride,
  0%|                                                                                                             | 0/30 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/administrator/Fooocus/modules/async_worker.py", line 896, in worker
    handler(task)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/async_worker.py", line 803, in handler
    imgs = pipeline.process_diffusion(
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/default_pipeline.py", line 362, in process_diffusion
    sampled_latent = core.ksampler(
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/core.py", line 308, in ksampler
    samples = ldm_patched.modules.sample.sample(model,
  File "/home/administrator/Fooocus/ldm_patched/modules/sample.py", line 100, in sample
    samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 712, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/sample_hijack.py", line 157, in sample_hacked
    samples = sampler.sample(model_wrap, sigmas, extra_args, callback_wrap, noise, latent_image, denoise_mask, disable_pbar)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 557, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/k_diffusion/sampling.py", line 701, in sample_dpmpp_2m_sde_gpu
    return sample_dpmpp_2m_sde(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, eta=eta, s_noise=s_noise, noise_sampler=noise_sampler, solver_type=solver_type)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/k_diffusion/sampling.py", line 613, in sample_dpmpp_2m_sde
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/patch.py", line 321, in patched_KSamplerX0Inpaint_forward
    out = self.inner_model(x, sigma,
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 271, in forward
    return self.apply_model(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 268, in apply_model
    out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed)
  File "/home/administrator/Fooocus/modules/patch.py", line 237, in patched_sampling_function
    positive_x0, negative_x0 = calc_cond_uncond_batch(model, cond, uncond, x, timestep, model_options)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 222, in calc_cond_uncond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
  File "/home/administrator/Fooocus/ldm_patched/modules/model_base.py", line 85, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/patch.py", line 404, in patched_unet_forward
    h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/diffusionmodules/openaimodel.py", line 43, in forward_timestep_embed
    x = layer(x, context, transformer_options)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 613, in forward
    x = block(x, context=context[i], transformer_options=transformer_options)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 440, in forward
    return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/diffusionmodules/util.py", line 189, in checkpoint
    return func(*inputs)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 500, in _forward
    n = self.attn1(n, context=context_attn1, value=value_attn1)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 392, in forward
    out = optimized_attention(q, k, v, self.heads)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 325, in attention_pytorch
    out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 622.00 MiB. GPU 0 has a total capacity of 11.17 GiB of which 165.25 MiB is free. Including non-PyTorch memory, this process has 11.01 GiB memory in use. Of the allocated memory 10.36 GiB is allocated by PyTorch, and 215.58 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Total time: 14.62 seconds
mashb1t commented 8 months ago

@giuseppelibrandi please run these commands:

git checkout 2.1.865
python3 launch.py
giuseppelibrandi commented 8 months ago

this is what i get

python3 launch.py
Note: switching to '2.1.865'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 1c999be Merge pull request #2229 from lllyasviel/develop
[System ARGV] ['launch.py']
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Fooocus version: 2.1.865
Failed to load config key: {"path_checkpoints": ["/home/administrator/Fooocus/models/checkpoints"]} is invalid or does not exist; will use {"path_checkpoints": "../models/checkpoints/"} instead.
Failed to load config key: {"path_loras": ["/home/administrator/Fooocus/models/loras"]} is invalid or does not exist; will use {"path_loras": "../models/loras/"} instead.
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
Total VRAM 11441 MB, total RAM 64264 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 Tesla K80 : native
VAE dtype: torch.float32
Using pytorch cross attention
Refiner unloaded.
model_type EPS
UNet ADM Dimension 2816
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
extra {'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
Base model loaded: /home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [/home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors].
Loaded LoRA [/home/administrator/Fooocus/models/loras/sd_xl_offset_example-lora_1.0.safetensors] for UNet [/home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cpu, use_fp16 = False.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 7457204670950791403
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] a dog, intricate, elegant, highly detailed, wonderful colors, warm light, sharp focus, majestic, epic composition, joyful, magical atmosphere, cinematic, new classic, dynamic, symmetry, fine detail, enhanced, clear, crisp, artistic, color, perfect, colorful, illuminated, beautiful, great, pure, true, full, creative, vibrant, loving, healthy
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding negative #1 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 7.44 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 2.80 seconds
  0%|                                                                                                             | 0/30 [00:00<?, ?it/s]/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Applied workaround for CuDNN issue, install nvrtc.so (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:80.)
  return F.conv2d(input, weight, bias, self.stride,
  0%|                                                                                                             | 0/30 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/administrator/Fooocus/modules/async_worker.py", line 822, in worker
    handler(task)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/async_worker.py", line 753, in handler
    imgs = pipeline.process_diffusion(
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/default_pipeline.py", line 361, in process_diffusion
    sampled_latent = core.ksampler(
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/core.py", line 313, in ksampler
    samples = ldm_patched.modules.sample.sample(model,
  File "/home/administrator/Fooocus/ldm_patched/modules/sample.py", line 100, in sample
    samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 712, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/sample_hijack.py", line 157, in sample_hacked
    samples = sampler.sample(model_wrap, sigmas, extra_args, callback_wrap, noise, latent_image, denoise_mask, disable_pbar)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 557, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/k_diffusion/sampling.py", line 701, in sample_dpmpp_2m_sde_gpu
    return sample_dpmpp_2m_sde(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, eta=eta, s_noise=s_noise, noise_sampler=noise_sampler, solver_type=solver_type)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/k_diffusion/sampling.py", line 613, in sample_dpmpp_2m_sde
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/patch.py", line 314, in patched_KSamplerX0Inpaint_forward
    out = self.inner_model(x, sigma,
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 271, in forward
    return self.apply_model(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 268, in apply_model
    out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed)
  File "/home/administrator/Fooocus/modules/patch.py", line 229, in patched_sampling_function
    positive_x0, negative_x0 = calc_cond_uncond_batch(model, cond, uncond, x, timestep, model_options)
  File "/home/administrator/Fooocus/ldm_patched/modules/samplers.py", line 222, in calc_cond_uncond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
  File "/home/administrator/Fooocus/ldm_patched/modules/model_base.py", line 85, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/modules/patch.py", line 398, in patched_unet_forward
    h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/diffusionmodules/openaimodel.py", line 43, in forward_timestep_embed
    x = layer(x, context, transformer_options)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 613, in forward
    x = block(x, context=context[i], transformer_options=transformer_options)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 440, in forward
    return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/diffusionmodules/util.py", line 189, in checkpoint
    return func(*inputs)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 500, in _forward
    n = self.attn1(n, context=context_attn1, value=value_attn1)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 392, in forward
    out = optimized_attention(q, k, v, self.heads)
  File "/home/administrator/Fooocus/ldm_patched/ldm/modules/attention.py", line 325, in attention_pytorch
    out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 622.00 MiB. GPU 0 has a total capacity of 11.17 GiB of which 165.25 MiB is free. Including non-PyTorch memory, this process has 11.01 GiB memory in use. Of the allocated memory 10.36 GiB is allocated by PyTorch, and 215.58 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Total time: 11.03 seconds
mashb1t commented 8 months ago

=> this is not an issue with 2.1.865, but something changed on your setup or you have insufficient resources. Please check if adding --attention-split to args helps or try other attention modes (can be found in the readme).

giuseppelibrandi commented 8 months ago

With the --attention-split it works but it take about 400 seconds to generate a image, is it right?

V2.2.0

(venv_fooocus) administrator@giuseppelibrandi:~/Fooocus$ python3 launch.py --attention-split
[System ARGV] ['launch.py', '--attention-split']
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Fooocus version: 2.2.0
Total VRAM 11441 MB, total RAM 64264 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 Tesla K80 : native
VAE dtype: torch.float32
Using split optimization for cross attention
Refiner unloaded.
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
model_type EPS
UNet ADM Dimension 2816
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
extra {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.text_projection'}
Base model loaded: /home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] for model [/home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors].
Loaded LoRA [/home/administrator/Fooocus/models/loras/sd_xl_offset_example-lora_1.0.safetensors] for UNet [/home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cpu, use_fp16 = False.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
Started worker with PID 2643
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ControlNet Softness = 0.25
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 8342520868371395375
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] a dog, intricate, elegant, highly detailed, sharp focus, beautiful, professional still, cinematic, background bright colors, inspired, designed, rich deep vivid color, perfect complex composition, dynamic light, precise, friendly, charismatic, cute, strong, innocent, pretty, best, novel, romantic, new, charming, attractive, great, contemporary, artistic, winning, gorgeous
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding negative #1 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 7.47 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 2.78 seconds
  0%|                                                                                                                                            | 0/30 [00:00<?, ?it/s]/home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Applied workaround for CuDNN issue, install nvrtc.so (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:80.)
  return F.conv2d(input, weight, bias, self.stride,
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [07:06<00:00, 14.20s/it]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.68 seconds
Image generated with private log at: /home/administrator/Fooocus/outputs/2024-03-04/log.html
Generating and saving time: 441.07 seconds
Total time: 448.59 seconds

V2.1.865


(venv_fooocus) administrator@giuseppelibrandi:~/Fooocus$ python3 launch.py --attention-split
[System ARGV] ['launch.py', '--attention-split']
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Fooocus version: 2.1.865
Failed to load config key: {"path_checkpoints": ["/home/administrator/Fooocus/models/checkpoints"]} is invalid or does not exist; will us                               e {"path_checkpoints": "../models/checkpoints/"} instead.
Failed to load config key: {"path_loras": ["/home/administrator/Fooocus/models/loras"]} is invalid or does not exist; will use {"path_lor                               as": "../models/loras/"} instead.
Running on local URL:  http://127.0.0.1:7865

To create a public link, set `share=True` in `launch()`.
Total VRAM 11441 MB, total RAM 64264 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 Tesla K80 : native
VAE dtype: torch.float32
Using split optimization for cross attention
Refiner unloaded.
model_type EPS
UNet ADM Dimension 2816
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
extra {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model                               .clip_l.text_projection'}
Base model loaded: /home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors
Request to load LoRAs [['sd_xl_offset_example-lora_1.0.safetensors', 0.1], ['None', 1.0], ['None', 1.0], ['None', 1.0], ['None', 1.0]] fo                               r model [/home/administrator/Fooocus/models/checkpoints/juggernautXL_v8Rundiffusion.safetensors].
Loaded LoRA [/home/administrator/Fooocus/models/loras/sd_xl_offset_example-lora_1.0.safetensors] for UNet [/home/administrator/Fooocus/mo                               dels/checkpoints/juggernautXL_v8Rundiffusion.safetensors] with 788 keys at weight 0.1.
Fooocus V2 Expansion: Vocab with 642 words.
Fooocus Expansion engine loaded for cpu, use_fp16 = False.
Requested to load SDXLClipModel
Requested to load GPT2LMHeadModel
Loading 2 new models
App started successful. Use the app with http://127.0.0.1:7865/ or 127.0.0.1:7865
[Parameters] Adaptive CFG = 7
[Parameters] Sharpness = 2
[Parameters] ADM Scale = 1.5 : 0.8 : 0.3
[Parameters] CFG = 4.0
[Parameters] Seed = 2224316946390489955
[Parameters] Sampler = dpmpp_2m_sde_gpu - karras
[Parameters] Steps = 30 - 15
[Fooocus] Initializing ...
[Fooocus] Loading models ...
Refiner unloaded.
[Fooocus] Processing prompts ...
[Fooocus] Preparing Fooocus text #1 ...
[Prompt Expansion] a dog, dramatic light, gorgeous background, full detail, dynamic composition, creative, vibrant colors, cozy atmospher                               e, cinematic, complex, highly detailed, professional, joyful, extremely inspirational, beautiful, stunning, symmetry, clear, artistic, sh                               arp, best, romantic, contemporary, futuristic, trendy, novel, epic, surreal, hopeful, exciting, thought, iconic, fine
[Fooocus] Encoding positive #1 ...
[Fooocus] Encoding negative #1 ...
[Parameters] Denoising Strength = 1.0
[Parameters] Initial Latent shape: Image Space (896, 1152)
Preparation time: 7.47 seconds
[Sampler] refiner_swap_method = joint
[Sampler] sigma_min = 0.0291671771556139, sigma_max = 14.614643096923828
Requested to load SDXL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 2.79 seconds
  0%|                                                                                                             | 0/30 [00:00<?, ?it/s]                               /home/administrator/Fooocus/venv_fooocus/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Applied workaround for C                               uDNN issue, install nvrtc.so (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:80.)
  return F.conv2d(input, weight, bias, self.stride,
100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [07:06<00:00, 14.20s/it]
Requested to load AutoencoderKL
Loading 1 new model
[Fooocus Model Management] Moving model(s) has taken 1.68 seconds
Image generated with private log at: /home/administrator/Fooocus/outputs/2024-03-04/log.html
Generating and saving time: 441.07 seconds
Total time: 455.68 seconds ```
mashb1t commented 8 months ago

The gist is that --attention-split differs in how resource allocation is done, which decreases generation time, but allows execution with less VRAM compared to standard.

DSV68609 commented 7 months ago

I have a question can Fooocus run on NVIDIA GForce MX130 GPU?

I have an error like this: in Runtime RuntimeError: The NVIDIA driver on your system is too old (found version 11040). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver.

mashb1t commented 7 months ago

@DSV68609 No, at least not until you update your nvidia driver and check cuda support for your card.

mashb1t commented 7 months ago

Closing as solved & not reproducible (OOM).