search

ToTheBeginning / PuLID

[NeurIPS 2024] Official code for PuLID: Pure and Lightning ID Customization via Contrastive Alignment
Apache License 2.0
2.72k stars 189 forks source link

Is 24G VRAM not enough? torch.OutOfMemoryError: CUDA out of memory #110

Open fahadshery opened 1 month ago

fahadshery commented 1 month ago

Hi,

I have a 24G VRAM Nvidia Tesla P40 card. I am getting the following error:

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 144.00 MiB. GPU 0 has a total capacity of 23.87 GiB of which 138.62 MiB is free. Process 13728 has 23.73 GiB memory in use. Of the allocated memory 23.55 GiB is allocated by PyTorch, and 18.42 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

Any solutions?

fahadshery commented 1 month ago

just noticed #65

fahadshery commented 1 month ago

I'm using a 24G card with compute capability of 6.1. Could we adapt to this?

ToTheBeginning commented 1 month ago

Please refer to the instruction here; even a 12GB graphics card can run PuLID-FLUX.

fahadshery commented 1 month ago

Please refer to the instruction here; even a 12GB graphics card can run PuLID-FLUX.

I ran it using: 

python app_flux.py --offload --fp8

But I am getting the compute capability error. Your application expects GPU CC to be above 8.0...Whilst majority of data centres and enterprise is still running before 6.0. It would be great if your application could run on these GPUs instead of GPUs with CC above 8