[Bug]: When using ControlNet Union with "Low VRAM" checked, I get the error: "Expected all tensors to be on the same device..." (Details in thread.) #16568
[ ] The issue exists after disabling all extensions
[ ] The issue exists on a clean installation of webui
[X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[ ] The issue exists in the current version of the webui
[X] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet
What happened?
I want to do inpainting, assisted by ControlNet Union (Depth and LineArt) and IPAdapter (FaceID); with the IPAdapter LoRA, and another LoRA for style, alongside SDXL (the model doesn't really matter, because I get it with both, but they are RealVis, and FaeTastic).
I can't load both the instances of Union and IPAdapter, the LoRAs, and SDXL in VRAM, because I run out. So, I usually check the "Low VRAM" option in ControlNet to lower memory usage, and prevent overflowing the graphics memory in the shared space (doesn't crash, but slows down tremendously; not relevant to the issue).
When I have "Low VRAM" checked, the program gives me an error message, and breaks. It doesn't crash, just breaks execution. The Error message is: "RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)"
(Details in the error logs.)
With LowVRAM unchecked, the program works.
If there's anything that I've missed, please, let me know. I'll try to provide as much info as I can.
Steps to reproduce the problem
Do anything related to inpainting.
Load ControlNet Union with both Depth and Lineart.
Preprocess the ControlNet input image, and set it as the input image. Preprocessor doesn't seem relevant - I just preprocess, drag to input, and set preprocessor to "None."
Either add an IPAdapter instance and check the "Low VRAM" option, or just check the "Low VRAM" option on either / both ControlNet Union tabs. It breaks either way.
Click "Generate" with your favorite prompt.
What should have happened?
It should just do the inpainting. Very preferably without breaking execution before it's done!
Checklist
What happened?
I want to do inpainting, assisted by ControlNet Union (Depth and LineArt) and IPAdapter (FaceID); with the IPAdapter LoRA, and another LoRA for style, alongside SDXL (the model doesn't really matter, because I get it with both, but they are RealVis, and FaeTastic). I can't load both the instances of Union and IPAdapter, the LoRAs, and SDXL in VRAM, because I run out. So, I usually check the "Low VRAM" option in ControlNet to lower memory usage, and prevent overflowing the graphics memory in the shared space (doesn't crash, but slows down tremendously; not relevant to the issue).
When I have "Low VRAM" checked, the program gives me an error message, and breaks. It doesn't crash, just breaks execution. The Error message is: "RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)" (Details in the error logs.)
With LowVRAM unchecked, the program works.
If there's anything that I've missed, please, let me know. I'll try to provide as much info as I can.
Steps to reproduce the problem
What should have happened?
It should just do the inpainting. Very preferably without breaking execution before it's done!
What browsers do you use to access the UI ?
Microsoft Edge
Sysinfo
sysinfo-2024-10-19-16-14.json
Console logs
Additional information
No response