out of memory

[Songssx]

I was able to use this repository almost a month ago. My computer has 6GB of graphics memory and 16GB of memory, but when I tried to use it again yesterday and today, I reported an error of insufficient graphics memory. Have you adjusted these settings.

[kijai]

I didn't change anything, but it seems that this just doesn't currently work with pytorch attention, not sure if it always was like that or with newer pytorch versions only. With xformers doing 1024x1024 image takes ~5GB, and without ~15GB. While I improved general memory use it would still require you to use xformers, if you weren't already.

[Songssx]

I didn't change anything, but it seems that this just doesn't currently work with pytorch attention, not sure if it always was like that or with newer pytorch versions only. With xformers doing 1024x1024 image takes ~5GB, and without ~15GB. While I improved general memory use it would still require you to use xformers, if you weren't already.

Thank you so much for your response! I tried reinstalling xformers and resolved the related error issues. This not only solved a problem for me but also expanded my knowledge. I really appreciate your guidance!

[burritotrex]

EDIT: Fixed it, xformers indeed was the problem - I reinstalled it according to the link provided in the message. And updated comfyui via the "update_comfyui_and_python_dependencies.bat" file - now everything works !

Hey, dont know if this is exactly related to this thread , but: I can run the DepthFM node once and it gives me one result (which are insanely good), also with different parameters and models used (fp16, original ckpt) - but afterwards it keeps throwing the same out of memory error (see below) - had the "no module 'xformers'" error before - installed it on the portable version, now during start it says:

WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for: PyTorch 2.3.0+cu121 with CUDA 1201 (you have 2.2.2+cu121) Python 3.11.9 (you have 3.11.8) Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers) Memory-efficient attention, SwiGLU, sparse and more won't be available. Set XFORMERS_MORE_DETAILS=1 for more details xformers version: 0.0.26.post1

`Error occurred when executing Depth_fm:

Allocation on device 0 would exceed allowed memory. (out of memory) Currently allocated : 18.53 GiB Requested : 14.89 GiB Device limit : 23.99 GiB Free (according to CUDA): 0 bytes PyTorch limit (set by user-supplied memory fraction) : 17179869184.00 GiB

File "P:\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\nodes.py", line 81, in process sub_images = self.model.predict_depth(images[start_idx:start_idx+per_batch], num_steps=steps, ensemble_size=ensemble_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\dfm.py", line 117, in predict_depth return self.forward(ims, num_steps, ensemble_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\dfm.py", line 97, in forward depth_z = self.generate(x_source, num_steps=num_steps, context=context, context_ca=conditioning) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\dfm.py", line 64, in generate ode_results = odeint(ode_fn, z, t, ode_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchdiffeq_impl\odeint.py", line 79, in odeint solution = solver.integrate(t) ^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchdiffeq_impl\solvers.py", line 114, in integrate dy, f0 = self._step_func(self.func, t0, dt, t1, y0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchdiffeq_impl\fixed_grid.py", line 10, in _step_func f0 = func(t0, y0, perturb=Perturb.NEXT if self.perturb else Perturb.NONE) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchdiffeq_impl\misc.py", line 197, in forward return self.base_func(t, y) ^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\dfm.py", line 47, in ode_fn return self.model(x=x, t=t, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\openaimodel.py", line 837, in forward h = module(h, emb, context_ca) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\openaimodel.py", line 85, in forward x = layer(x, context) ^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\attention.py", line 371, in forward x = block(x, context=context[i]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\attention.py", line 285, in forward return checkpoint( ^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\util.py", line 39, in checkpoint return func(inputs) ^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\attention.py", line 291, in _forward self.attn1( File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\attention.py", line 179, in forward out = F.scaled_dot_product_attention(q, k, v, scale=(math.log(n_tokens) / math.log(n_tokens*4) / self.dim_head)**0.5 if is_self_attention else None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^`

Is xformer still the issue? Or does it only work with specific set parameters?

[kijai]

EDIT: Fixed it, xformers indeed was the problem - I reinstalled it according to the link provided in the message. And updated comfyui via the "update_comfyui_and_python_dependencies.bat" file - now everything works !

Hey, dont know if this is exactly related to this thread , but: I can run the DepthFM node once and it gives me one result (which are insanely good), also with different parameters and models used (fp16, original ckpt) - but afterwards it keeps throwing the same out of memory error (see below) - had the "no module 'xformers'" error before - installed it on the portable version, now during start it says:

WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for: PyTorch 2.3.0+cu121 with CUDA 1201 (you have 2.2.2+cu121) Python 3.11.9 (you have 3.11.8) Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers) Memory-efficient attention, SwiGLU, sparse and more won't be available. Set XFORMERS_MORE_DETAILS=1 for more details xformers version: 0.0.26.post1

`Error occurred when executing Depth_fm:

Allocation on device 0 would exceed allowed memory. (out of memory) Currently allocated : 18.53 GiB Requested : 14.89 GiB Device limit : 23.99 GiB Free (according to CUDA): 0 bytes PyTorch limit (set by user-supplied memory fraction) : 17179869184.00 GiB

File "P:\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\nodes.py", line 81, in process sub_images = self.model.predict_depth(images[start_idx:start_idx+per_batch], num_steps=steps, ensemble_size=ensemble_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\dfm.py", line 117, in predict_depth return self.forward(ims, num_steps, ensemble_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\dfm.py", line 97, in forward depth_z = self.generate(x_source, num_steps=num_steps, context=context, context_ca=conditioning) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\dfm.py", line 64, in generate ode_results = odeint(ode_fn, z, t, ode_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchdiffeq_impl\odeint.py", line 79, in odeint solution = solver.integrate(t) ^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchdiffeq_impl\solvers.py", line 114, in integrate dy, f0 = self._step_func(self.func, t0, dt, t1, y0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchdiffeq_impl\fixed_grid.py", line 10, in _step_func f0 = func(t0, y0, perturb=Perturb.NEXT if self.perturb else Perturb.NONE) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchdiffeq_impl\misc.py", line 197, in forward return self.base_func(t, y) ^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\dfm.py", line 47, in ode_fn return self.model(x=x, t=t, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\openaimodel.py", line 837, in forward h = module(h, emb, context_ca) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\openaimodel.py", line 85, in forward x = layer(x, context) ^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\attention.py", line 371, in forward x = block(x, context=context[i]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\attention.py", line 285, in forward return checkpoint( ^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\util.py", line 39, in checkpoint return func(inputs) ^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\attention.py", line 291, in _forward self.attn1( File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(_args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "P:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-depth-fm\depthfm\unet\attention.py", line 179, in forward out = F.scaled_dot_product_attention(q, k, v, scale=(math.log(n_tokens) / math.log(n_tokens_4) / self.dim_head)0.5 if is_self_attention else None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^`

Is xformer still the issue? Or does it only work with specific set parameters?

Latest xformers expects torch 2.3.0, you have 2.2.0. I would recommend updating torch, though some older version of xformers would also work, I just don't know the exact version for torch 2.2.0.

[wes-kay]

torch.OutOfMemoryError: Allocation on device

I'm on a 4090, with comfy ui nightly (torch 2.5) trying to depth on a 1024x1024 Any ideas?