Hey, I've been trying to figure out why a docker of the huggingface space demo running through WSL went over 24 GB of my VRAM even though the colab and the hugging face site itself seemed to be ok (same parameters/length ofc, even with kv cache off).
While playing around, I've discovered that if I press "load models" more than once, to change a setting for example, the vram total usage went up and stayed up (even if I let's say removed a whisper model, so it should decrease). So could it be a bug that when running multiple prompts causes it to not unload a model hence the steep VRAM increase?
Hopefully it's useful and that it's an easy fix whenever you have the time if you can (I wish I could help more but I've literally downloaded linux first time today just for this).
Hey, I've been trying to figure out why a docker of the huggingface space demo running through WSL went over 24 GB of my VRAM even though the colab and the hugging face site itself seemed to be ok (same parameters/length ofc, even with kv cache off).
While playing around, I've discovered that if I press "load models" more than once, to change a setting for example, the vram total usage went up and stayed up (even if I let's say removed a whisper model, so it should decrease). So could it be a bug that when running multiple prompts causes it to not unload a model hence the steep VRAM increase?
Hopefully it's useful and that it's an easy fix whenever you have the time if you can (I wish I could help more but I've literally downloaded linux first time today just for this).
Thanks!