techniqs
commented
1 week ago I fixed this problem via downgrading pytorch from 2.5.0 to 2.4.1 looks like its something about pytorch breaking SDPA where i got it from
Open dobby-hsb opened 1 month ago
techniqs
commented
1 week ago I fixed this problem via downgrading pytorch from 2.5.0 to 2.4.1 looks like its something about pytorch breaking SDPA where i got it from
I'm getting a cuDNN Frontend error and I can't seem to resolve it.
ComfyUI Error Report
Error Details
Exception Message: cuDNN Frontend error: [cudnn_frontend] Error: No execution plans support the graph.
Stack Trace
2024-10-18 18:11:20,538 - root - ERROR - Traceback (most recent call last): File "/projects/stability_matrix/Data/Packages/ComfyUI/execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/projects/stability_matrix/Data/Packages/ComfyUI/execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "/projects/stability_matrix/Data/Packages/ComfyUI/execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "/projects/stability_matrix/Data/Packages/ComfyUI/execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) File "/projects/stability_matrix/Data/Packages/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/nodes.py", line 264, in sample prompt_embeds, prompt_attention_mask, pooled_prompt_embeds = text_encoder(positive_prompt, device) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(args, kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/modeling_text_encoder.py", line 138, in forward prompt_embeds, prompt_attention_mask, pooled_prompt_embeds = self.encode_prompt(input_prompts, 1, clip_skip=None, device=device) File "/projects/stability_matrix/Data/Packages/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/modeling_text_encoder.py", line 113, in encode_prompt pooled_prompt_embed = self._get_clip_prompt_embeds( File "/projects/stability_matrix/Data/Packages/ComfyUI/custom_nodes/ComfyUI-PyramidFlowWrapper/pyramid_dit/modeling_text_encoder.py", line 98, in _get_clip_prompt_embeds prompt_embeds = text_encoder(text_input_ids.to(device), output_hidden_states=True) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, *kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 1413, in forward text_outputs = self.text_model( File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 901, in forward encoder_outputs = self.encoder( File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(args, kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 824, in forward layer_outputs = encoder_layer( File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, *kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 559, in forward hidden_states, attn_weights = self.self_attn( File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/projects/stability_matrix/Data/Packages/ComfyUI/venv/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 491, in forward attn_output = torch.nn.functional.scaled_dot_product_attention( RuntimeError: cuDNN Frontend error: [cudnn_frontend] Error: No execution plans support the graph.