search

deep-floyd / IF

Other
7.69k stars 501 forks source link

CUDA out of memory. #85

Closed realcarlos closed 1 year ago

realcarlos commented 1 year ago

However I am using a station with 4 x A100(40G)

if_I = IFStageI('/IF/deepfloyd-if/IF-I-XL-v1.0', device='cuda:0') if_II = IFStageII('/IF/deepfloyd-if/IF-II-L-v1.0', device='cuda:1') if_III = StableStageIII('/IF/deepfloyd-if/stable-diffusion-x4-upscaler', device='cuda:2') t5 = T5Embedder(device="cuda:3")

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 39.39 GiB total capacity; 29.37 GiB already allocated; 6.90 GiB free; 30.95 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

realcarlos commented 1 year ago

solved by adding "os.environ["FORCE_MEM_EFFICIENT_ATTN"] = "1""

YeHaijia commented 1 year ago

对我有用，赞

404-xianjin commented 1 year ago

hi，I got the same error, my error code is as follows, torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 GiB (GPU 0; 23.99 GiB total capacity; 17.99 GiB already allocated; 3.23 GiB free; 18.43 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

After seeing your solution, I don't know where to set this parameter? Can you tell me? Thanks @realcarlos

404-xianjin commented 1 year ago

大哥，我点开你的主页发现你也在北京，我直接 image 目前看好像可以了。

Delicious-Bitter-Melon commented 1 year ago

大哥，我点开你的主页发现你也在北京，我直接 image 目前看好像可以了。

I have the same problem using the 3090. Although I set the ""os.environ["FORCE_MEM_EFFICIENT_ATTN"] = "1""", I still have the problem. Do you also use the 3090 with the 24GB?