search

kijai / ComfyUI-DynamiCrafterWrapper

Wrapper to use DynamiCrafter models in ComfyUI
Other
631 stars 26 forks source link

Error occurred when executing DynamiCrafterI2V: #64

Open XTRMsavage opened 5 months ago

XTRMsavage commented 5 months ago

Hi, I am getting this error, what's the cause ? is it XFORMERS?

image

Error occurred when executing DynamiCrafterI2V:

No operator found for memory_efficient_attention_forward with inputs: query : shape=(80, 7168, 1, 64) (torch.float16) key : shape=(80, 7168, 1, 64) (torch.float16) value : shape=(80, 7168, 1, 64) (torch.float16) attn_bias : p : 0.0 decoderF is not supported because: xFormers wasn't build with CUDA support attn_bias type is operator wasn't built - see python -m xformers.info for more info flshattF@0.0.0 is not supported because: xFormers wasn't build with CUDA support operator wasn't built - see python -m xformers.info for more info cutlassF is not supported because: xFormers wasn't build with CUDA support operator wasn't built - see python -m xformers.info for more info smallkF is not supported because: max(query.shape[-1] != value.shape[-1]) > 32 xFormers wasn't build with CUDA support dtype=torch.float16 (supported: {torch.float32}) operator wasn't built - see python -m xformers.info for more info unsupported embed per head: 64

File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func res_value = old_func(final_args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\customnodes\ComfyUI-DynamiCrafterWrapper\nodes.py", line 454, in process samples, = ddim_sampler.sample(S=steps, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\samplers\ddim.py", line 119, in sample samples, intermediates = self.ddim_sampling(conditioning, size, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\samplers\ddim.py", line 192, in ddim_sampling outs = self.p_sample_ddim(img, cond, ts, index=index, use_original_steps=ddim_use_original_steps, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\samplers\ddim.py", line 228, in p_sample_ddim e_t_cond = self.model.apply_model(x, t, c, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\ddpm3d.py", line 574, in apply_model x_recon = self.model(x_noisy, t, cond, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\models\ddpm3d.py", line 736, in forward out = self.diffusion_model(xc, t, context=cc, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\networks\openaimodel3d.py", line 583, in forward h = module(h, emb, context=context, batch_size=b, frame_window_size=frame_window_size, frame_window_stride=frame_window_stride) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\networks\openaimodel3d.py", line 41, in forward x = layer(x, context) ^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\attention.py", line 311, in forward x = block(x, context=context, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\attention.py", line 246, in forward return checkpoint(self._forward, input_tuple, self.parameters(), self.checkpoint) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\common.py", line 94, in checkpoint return func(inputs) ^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\attention.py", line 250, in _forward x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None, mask=mask) + x ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DynamiCrafterWrapper\lvdm\modules\attention.py", line 182, in efficient_forward out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha__init.py", line 268, in memory_efficient_attention return _memory_efficient_attention( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha__init__.py", line 387, in _memory_efficient_attention return _memory_efficient_attention_forward( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha\init__.py", line 403, in _memory_efficient_attention_forward op = _dispatch_fw(inp, False) ^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 125, in _dispatch_fw return _run_priority_list( ^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_newbuilds\May\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 65, in _run_priority_list raise NotImplementedError(msg)

kijai commented 5 months ago

Yes, your xformers installation isn't compatible with your torch install. Possibly just reinstalling xformers could help, if not then re-installing xformers compatible torch version is required, for example latest stable 2.3.1 works with latest xformers.

This is all assuming you have nvidia GPU, xformers will not work otherwise and should simply be uninstalled.

Uninstall xformers: python_embeded\python.exe -m pip uninstall xformers Install latest version without dependencies: python_embeded\python.exe -m pip xformers --no-deps

If that doesn't help:

Uninstall current torch: python_embeded\python.exe -m pip uninstall torch torchvision torchaudio Install latest torch: python_embeded\python.exe -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

moon47usaco commented 5 months ago

This worked for me:

Uninstall both: pip uninstall xformers pip uninstall torch torchvision torchaudio

Reinstall both: Used xformers install from facebookresearch/xformers ::: pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121 pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Since the recommended command from fb page was pip3 i just used it for both. I'm fairly sure pip would have worked for both...

XTRMsavage commented 5 months ago

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Thanks, I got the xformers installed but torchvision fails due to it's file size. Is there a way I can download it separately and put in the right location?