Moondream node is failing to run with either version (moondream1 or moondream2). I have it running on another machine with the exact same nodes installed, same workflow, parameters identical. The only difference between these two machines is that this one has a 3090 and the working one has a 4090.
Both machines are running freshly built comfyUI portable
The output when running on the 3090 is as follows:
Error occurred when executing MoondreamQuery:
index is out of bounds for dimension with size 0
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-moondream\nodes.py", line 81, in process
answer = self.moondream.answer_question(image_embeds, question, self.tokenizer, max_new_tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-moondream\moondream\moondream.py", line 93, in answer_question
answer = self.generate(
^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-moondream\moondream\moondream.py", line 76, in generate
output_ids = self.text_model.generate(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\generation\utils.py", line 1914, in generate
result = self._sample(
^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\generation\utils.py", line 2651, in _sample
outputs = self(
^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-moondream\moondream\modeling_phi.py", line 1104, in forward
outputs = self.transformer(
^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-moondream\moondream\modeling_phi.py", line 959, in forward
layer_outputs = decoder_layer(
^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-moondream\moondream\modeling_phi.py", line 763, in forward
attn_outputs, self_attn_weights, present_key_value = self.mixer(
^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-moondream\moondream\modeling_phi.py", line 382, in forward
query_rot, key_rot = apply_rotary_pos_emb(
^^^^^^^^^^^^^^^^^^^^^
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-moondream\moondream\modeling_phi.py", line 214, in apply_rotary_pos_emb
cos = cos[position_ids].unsqueeze(unsqueeze_dim)
~~~^^^^^^^^^^^^^^
Moondream node is failing to run with either version (moondream1 or moondream2). I have it running on another machine with the exact same nodes installed, same workflow, parameters identical. The only difference between these two machines is that this one has a 3090 and the working one has a 4090.
Both machines are running freshly built comfyUI portable
The output when running on the 3090 is as follows: