CUDA out of memory

[linghu-cell]

I am NVIDIA GeForce RTX 3090 with 24GB of graphics memory, running at 1024 resolution, but the graphics memory is not enough

[ApolloRay]

I am NVIDIA GeForce RTX 3090 with 24GB of graphics memory, running at 1024 resolution, but the graphics memory is not enough

https://github.com/PixArt-alpha/PixArt-sigma/issues/19 , follow this issue.

[lawrence-cj]

Training or testing?

[linghu-cell]

培训还是测试？

testing

[linghu-cell]

Training or testing?

Run script: Python scripts/interface.py

[lawrence-cj]

Try these 18GB minimum with https://github.com/PixArt-alpha/PixArt-sigma#3-pixart-demo. 8GB with https://github.com/PixArt-alpha/PixArt-sigma/blob/master/asset/docs/pixart.md

[linghu-cell]

I have no problem running the reference code you provided. What is the reason for this?

eg: import torch from diffusers import Transformer2DModel from scripts.diffusers_patches import pixart_sigma_init_patched_inputs, PixArtSigmaPipeline

assert getattr(Transformer2DModel, '_init_patched_inputs', False), "Need to Upgrade diffusers: pip install git+https://github.com/huggingface/diffusers" setattr(Transformer2DModel, '_init_patched_inputs', pixart_sigma_init_patched_inputs) device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") weight_dtype = torch.float16

transformer = Transformer2DModel.from_pretrained( "PixArt-alpha/PixArt-Sigma-XL-2-1024-MS", subfolder='transformer', torch_dtype=weight_dtype, use_safetensors=True, ) pipe = PixArtSigmaPipeline.from_pretrained( "PixArt-alpha/pixart_sigma_sdxlvae_T5_diffusers", transformer=transformer, torch_dtype=weight_dtype, use_safetensors=True, ) pipe.to(device)

Enable memory optimizations.

pipe.enable_model_cpu_offload()

prompt = "A small cactus with a happy face in the Sahara desert." image = pipe(prompt).images[0] image.save("./catcus.png")

[lawrence-cj]

I don't get you. What's the problem?

[linghu-cell]

I have no problem running the code you provided. What is the reason for the same 1024 resolution? https://github.com/PixArt-alpha/PixArt-sigma?tab=readme-ov-file#2-integration-in-diffusers

[lawrence-cj]

No idea. Maybe diffusers have some optimization?