lshqqytiger
commented
1 year ago torch-directml can't release/collect/empty memory because its tensor implementation inherits OpaqueTensorImpl which can't have storage.
import torch
import torch_directml
device = torch_directml.device()
torch.tensor(1.0).storage() # [torch.storage.TypedStorage(dtype=torch.float32, device=cpu) of size 1]
torch.tensor(1.0).to(device).storage() # NotImplementedError: Cannot access storage of OpaqueTensorImplBut onnxruntime-directml partially does release memory. If you want something like that, you can use Olive+ONNX implementation instead.
cactus24556
Lamer217
GeneralVincent1
johiny
Is there an existing issue for this?
What happened?
I launch the webui from the webui-user.bat and it starts up normally except I notice once the webui is open in my browser my VRAM is filled about to 5GB out of my 8GB. It seems to be happening when it is trying to load the model. So when i try to generate a image it runs out of VRAM in the middle and the generation is stopped. This even happens on a fresh install of the webui where it uses the default model.
Also after trying to generate the image unsuccessfully the VRAM is not released at all. My VRAM is completely full until I shutdown the program.
Another issue is that in the webui I am unable to change which model the generator uses. I pick another model and it shows it processing it but it fails to change and also it takes up my VRAM and does not release any VRAM when it finishes.
Steps to reproduce the problem
What should have happened?
I should have enough VRAM to generate a 512x512 image without running out of VRAM. My VRAM should be clear if I'm not generating anything.
Sysinfo
sysinfo-2023-09-08-20-11.txt
What browsers do you use to access the UI ?
Mozilla Firefox
Console logs
Additional information
No response