There is a way to use flash attention in AMD Windows? RX580

[KillyTheNetTerminal]

[kijai]

I think there is, but that requirement being forced is actually a bug in the original code. I've added a workaround now and choosing other attention types will actually be allowed.

[patientx]

I think there is, but that requirement being forced is actually a bug in the original code. I've added a workaround now and choosing other attention types will actually be allowed.

update: possible fix for zluda. https://github.com/vladmandic/automatic/commit/f3b33c4c93712027778745e2e0595602dad093f6

both other options just quit without any errors on comfyui (using zluda), both gives this error using (directml)

`Error occurred when executing Florence2Run:

probability tensor contains either inf, nan or element < 0

File "D:\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "D:\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "D:\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(*slice_dict(input_data_all, i))) File "D:\ComfyUI\custom_nodes\ComfyUI-Florence2\nodes.py", line 179, in encode generated_ids = model.generate( File "C:\Users\aslan.cache\huggingface\modules\transformers_modules\Florence-2-base-ft\modeling_florence2.py", line 2790, in generate return self.language_model.generate( File "D:\ComfyUI\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) File "D:\ComfyUI\venv\lib\site-packages\transformers\generation\utils.py", line 1795, in generate result = self._beam_search( File "D:\ComfyUI\venv\lib\site-packages\transformers\generation\utils.py", line 2706, in _beam_search next_tokens = torch.multinomial(probs, num_samples=n_tokens_to_keep)`

on the other hand cpu works of course very very slow.

[KillyTheNetTerminal]

what is that error?

[Geeknasty]

I'm also seeing a quit without error in log when running this node. AMD Windows with Zluda.

[kijai]

I think there is, but that requirement being forced is actually a bug in the original code. I've added a workaround now and choosing other attention types will actually be allowed.

update: possible fix for zluda. vladmandic/automatic@f3b33c4

both other options just quit without any errors on comfyui (using zluda), both gives this error using (directml)

`Error occurred when executing Florence2Run:

probability tensor contains either inf, nan or element < 0

File "D:\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "D:\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "D:\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(*slice_dict(input_data_all, i))) File "D:\ComfyUI\custom_nodes\ComfyUI-Florence2\nodes.py", line 179, in encode generated_ids = model.generate( File "C:\Users\aslan.cache\huggingface\modules\transformers_modules\Florence-2-base-ft\modeling_florence2.py", line 2790, in generate return self.language_model.generate( File "D:\ComfyUI\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) File "D:\ComfyUI\venv\lib\site-packages\transformers\generation\utils.py", line 1795, in generate result = self._beam_search( File "D:\ComfyUI\venv\lib\site-packages\transformers\generation\utils.py", line 2706, in _beam_search next_tokens = torch.multinomial(probs, num_samples=n_tokens_to_keep)`

on the other hand cpu works of course very very slow.

That fix on the sd.NEXT is the exact one I had already applied.

This error happens within transformers so it doesn't have much to do with the node, does it happen with all dtypes though?

I don't have access to AMD GPU so I can't really troubleshoot this further.