search

basujindal / stable-diffusion

Optimized Stable Diffusion modified to run on lower GPU VRAM
Other
3.14k stars 469 forks source link

VRAM is full after finished a prompt with gradio and it forces me to restart service #131

Open konimaki2022 opened 2 years ago

konimaki2022 commented 2 years ago

After finishing the execution of a prompt in gradio, the cache of the VRAM is not released, forcing in systems with low VRAM to restart the execution of the service and reload the model, previous to launch a new prompt.

This happened to me with a NVIDIA GTX 1660 6GB.

It can be solved with torch.cuda.empty_cache()

basujindal commented 2 years ago

Hi, I am not able to reproduce the issue. Just to mention, nvidia-smi shows VRAM in use even if no tensor/object is the memory as torch reserves the memory for faster access.

konimaki2022 commented 2 years ago

Hello! VRAM was always around 80% after finishing the first prompt, and when trying to launch the next one, I got a CUDA memory allocation error forcing me to restart gradio every time.

I solved it after putting that line in: 

del samples_ddim
del x_sample
del x_samples_ddim
torch.cuda.empty_cache()

After that it allows me to run one prompt after another without having to restart it.