Always says "CUDA Out of Memory"

[phr00t]

I have a 12GB 4080 and I'm trying to run the 512x512 interpolation model on a 512x512 pair of images. It says it is requesting only 5GB of VRAM, which I should definitely have.

I'm trying to use the most basic workflow:

[kijai]

What dimension are the images? The node itself doesn't change any of that, you'd have to use some other node to downsize it first to 512p, it doesn't have to be exactly that though, just close.

[phr00t]

What dimension are the images? The node itself doesn't change any of that, you'd have to use some other node to downsize it first to 512p, it doesn't have to be exactly that though, just close.

Images are both 512x512

[hben35096]

Hi, I'm running on both computers and both get the same error：

/mnt/workspace/ComfyUI ComfyUI startup time: 2024-03-17 02:41:20.767899 Platform: Linux Python version: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] Python executable: /opt/conda/bin/python ** Log path: /mnt/workspace/ComfyUI/comfyui.log

Prestartup times for custom nodes: 0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-Manager

Total VRAM 22732 MB, total RAM 30179 MB xformers version: 0.0.23.post1 Set vram state to: NORMAL_VRAM Device: cuda:0 NVIDIA A10 : cudaMallocAsync VAE dtype: torch.bfloat16 Using xformers cross attention

Loading: ComfyUI-Manager (V2.10.1)

ComfyUI Revision: 2066 [f2fe635c] | Released on '2024-03-15'

Import times for custom nodes: 0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/AIGODLIKE-COMFYUI-TRANSLATION 0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-DynamiCrafterWrapper 0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-Manager 0.1 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite

Starting server

To see the GUI go to: [http://127.0.0.1:8188] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json FETCH DATA from: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-Manager/extension-node-map.json Client protocols [''] don’t overlap server-known ones () got prompt 2024-03-17 02:41:38.726666: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0. 2024-03-17 02:41:38.869058: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used. 2024-03-17 02:41:39.439351: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-03-17 02:41:39.439390: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-03-17 02:41:39.443115: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-03-17 02:41:39.751616: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used. 2024-03-17 02:41:39.752933: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2024-03-17 02:41:41.109307: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT [2024-03-17 02:41:42,820] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect) LatentVisualDiffusion: Running in v-prediction mode AE working on z of shape (1, 4, 32, 32) = 4096 dimensions. Loaded ViT-H-14 model config. Loading pretrained ViT-H-14 weights (laion2b_s32b_b79k). Loaded ViT-H-14 model config. Loading pretrained ViT-H-14 weights (laion2b_s32b_b79k). Client protocols [''] don’t overlap server-known ones ()

model checkpoint loaded. !!! Exception during processing !!! Traceback (most recent call last): File "/mnt/workspace/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/mnt/workspace/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/mnt/workspace/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/mnt/workspace/ComfyUI/custom_nodes/ComfyUI-DynamiCrafterWrapper/nodes.py", line 338, in process out_video = torch.cat(out, dim=0) RuntimeError: torch.cat(): expected a non-empty list of Tensors

Prompt executed in 113.28 seconds Client protocols [''] don’t overlap server-known ones () got prompt !!! Exception during processing !!! Traceback (most recent call last): File "/mnt/workspace/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/mnt/workspace/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/mnt/workspace/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/mnt/workspace/ComfyUI/custom_nodes/ComfyUI-DynamiCrafterWrapper/nodes.py", line 338, in process out_video = torch.cat(out, dim=0) RuntimeError: torch.cat(): expected a non-empty list of Tensors

[phr00t]

That doesn't look like the same error I'm getting...

[kijai]

What dimension are the images? The node itself doesn't change any of that, you'd have to use some other node to downsize it first to 512p, it doesn't have to be exactly that though, just close.

Images are both 512x512

That's really weird then, at that resolution my memory usage barely hits 8GB...

[phr00t]

What dimension are the images? The node itself doesn't change any of that, you'd have to use some other node to downsize it first to 512p, it doesn't have to be exactly that though, just close.

Images are both 512x512

That's really weird then, at that resolution my memory usage barely hits 8GB...

Good news! I just checked out the latest build and I no longer have this problem. No idea either, but closing as "resolved", ha. Thank you for the quick feedback! Looking forward to messing with this.

[kijai]

What dimension are the images? The node itself doesn't change any of that, you'd have to use some other node to downsize it first to 512p, it doesn't have to be exactly that though, just close.

Images are both 512x512

That's really weird then, at that resolution my memory usage barely hits 8GB...

Good news! I just checked out the latest build and I no longer have this problem. No idea either, but closing as "resolved", ha. Thank you for the quick feedback! Looking forward to messing with this.

Ok good to hear, it's very much work in progress and every update requires the node to be re-created currently, could be there was some commit I broke something.

[phr00t]

What dimension are the images? The node itself doesn't change any of that, you'd have to use some other node to downsize it first to 512p, it doesn't have to be exactly that though, just close.

Images are both 512x512

That's really weird then, at that resolution my memory usage barely hits 8GB...

Good news! I just checked out the latest build and I no longer have this problem. No idea either, but closing as "resolved", ha. Thank you for the quick feedback! Looking forward to messing with this.

Ok good to hear, it's very much work in progress and every update requires the node to be re-created currently, could be there was some commit I broke something.

Spoke too soon.... came back after trying to run it again :(

[phr00t]

It seems to give me this error the first time I try it, and then I rerun my workflow and then it "works". So, inconsistent error...

[phr00t]

I just want to confirm: when I get this error, simply just queuing the same exact workflow immediately after the error works fine.

[kijai]

There is something that stays loaded between runs even if tryin to force unloading/offloading to system RAM.. haven't figured it out yet.

[kijai]

I just want to confirm: when I get this error, simply just queuing the same exact workflow immediately after the error works fine.

It seems to work now, at least for me if keep_model_loaded is disabled it moves the unet to system memory and VRAM usage drops back to normal after the run.

[phr00t]

I just want to confirm: when I get this error, simply just queuing the same exact workflow immediately after the error works fine.

It seems to work now, at least for me if keep_model_loaded is disabled it moves the unet to system memory and VRAM usage drops back to normal after the run.

I'm still getting this error with keep_model_loaded set to false. I'll have the error popup when I first load my workflow, images are made from my KSampler and then fed into DynamiCrafter. I'll close out the error, retry the same workflow, and it works. Note that when it "works", it doesn't need to do the KSampler stuff as the images are already generated in ComfyUI and it knows it can skip that step. If I change the KSampler settings, like the seed, I WILL get this CUDA memory error again. So, it seems to be related to generating images before DynamicCrafter happens, maybe?

[kijai]

I just want to confirm: when I get this error, simply just queuing the same exact workflow immediately after the error works fine.

It seems to work now, at least for me if keep_model_loaded is disabled it moves the unet to system memory and VRAM usage drops back to normal after the run.

I'm still getting this error with keep_model_loaded set to false. I'll have the error popup when I first load my workflow, images are made from my KSampler and then fed into DynamiCrafter. I'll close out the error, retry the same workflow, and it works. Note that when it "works", it doesn't need to do the KSampler stuff as the images are already generated in ComfyUI and it knows it can skip that step. If I change the KSampler settings, like the seed, I WILL get this CUDA memory error again. So, it seems to be related to generating images before DynamicCrafter happens, maybe?

Oh yeah that's common error in comfy, happens with other workflows as well. I'm unsure why as there's robust memory management and in this node I'm always forcing cache clear at start too.

[VadimFedenko]

i have rtx4090, 24gb vram. for some reason the node only allocates 8gb, so i can't do higher than 512x512

[kijai]

i have rtx4090, 24gb vram. for some reason the node only allocates 8gb, so i can't do higher than 512x512

I don't understand what you mean by that?

[yorkane]

pip3 install xformers

to fix this issue

[kijai]

It does indeed seem like this currently requires xformers.

[phr00t]

Installing xformers per new instructions on the github readme resolved this. Thank you!