Documentation - what is the GPU requirement to run inference

wgong commented 1 month ago

running scripts/inference.py throws the following error

    return forward_call(*args, **kwargs)
  File "/home/gongai/projects/wgong/PromptFix/./stable_diffusion/ldm/modules/attention.py", line 192, in forward
    sim = einsum('b i d, b j d -> b i j', q, k) * self.scale
  File "/home/gongai/anaconda3/envs/promptfix/lib/python3.10/site-packages/torch/functional.py", line 377, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 6.19 GiB. GPU 0 has a total capacty of 7.74 GiB of which 1.64 GiB is free. Including non-PyTorch memory, this process has 5.58 GiB memory in use. Of the allocated memory 5.03 GiB is allocated by PyTorch, and 430.07 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Please update README with minimum required GPT vRAM,

wgong commented 1 month ago

I am using NVIDIA GeForce RTX 4060 with 8.1GB vRAM, after reducing image resolution and batch-size, I am able to run inference on those example images. Going to submit PR later

deepfree2023 commented 3 weeks ago

It's very vram consuming, peak vram usage was about 35GB just running the example images, I could run it just because the shared vram was large enough.

smart4654154 commented 1 week ago

It's very vram consuming, peak vram usage was about 35GB just running the example images, I could run it just because the shared vram was large enough.

Are you using Win's shared vram?thank you

yeates / PromptFix

Documentation - what is the GPU requirement to run inference #6