Issue with moondream2

[Eagle07-Sudo]

I am getting an issue while running moondream2 on Windows with an RTX 4090 GPU

FETCH DATA from: C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json

read_workflow_json_files_all C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-mixlab-nodes\app\

FETCH DATA from: C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json ERROR:asyncio:Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)> Traceback (most recent call last): File "asyncio\events.py", line 80, in _run File "asyncio\proactor_events.py", line 165, in _call_connection_lost ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host ERROR:asyncio:Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)> Traceback (most recent call last): File "asyncio\events.py", line 80, in _run File "asyncio\proactor_events.py", line 165, in _call_connection_lost ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func res_value = old_func(final_args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_VLM_nodes\nodes\moondream2.py", line 68, in moondream2_generate_predictions response = self.predictor.generate_predictions(temp_path, text_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_VLM_nodes\nodes\moondream2.py", line 33, in generate_predictions generated_text = self.model.answer_question(enc_image, question, self.tokenizer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\moondream.py", line 92, in answer_question answer = self.generate( ^^^^^^^^^^^^^^ File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\moondream.py", line 76, in generate output_ids = self.text_model.generate( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\generation\utils.py", line 1544, in generate return self.greedy_search( ^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\generation\utils.py", line 2404, in greedy_search outputs = self( ^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\modeling_phi.py", line 709, in forward hidden_states = self.transformer( ^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\modeling_phi.py", line 675, in forward else func(args) ^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\modeling_phi.py", line 541, in forward attn_outputs = self.mixer( ^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\modeling_phi.py", line 514, in forward attn_output_function(x, past_key_values, attention_mask) File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\modeling_phi.py", line 494, in _forward_cross_attn return attn_func( ^^^^^^^^^^ File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\modeling_phi.py", line 491, in else lambda fn, args, kwargs: fn(*args, kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\amp\autocast_mode.py", line 16, in decorate_autocast return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\amp\autocast_mode.py", line 16, in decorate_autocast return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a.elmoussawi.cache\huggingface\modules\transformers_modules\files_for_moondream2\modeling_phi.py", line 318, in forward padding_mask.maskedfill(key_padding_mask, 0.0) RuntimeError: The expanded size of the tensor (748) must match the existing size (749) at non-singleton dimension 1. Target sizes: [1, 748]. Tensor sizes: [1, 749]

[gokayfem]

im aware of this problem on some cards it does not work, i will fix this today.

[wibur0620]

There's not enough video memory.

[gokayfem]

i down graded the transformers to 4.36.2 this solved the problem.