search

Hangover3832 / ComfyUI-Hangover-Moondream

Moondream is a lightweight multimodal large language model
https://github.com/Hangover3832/ComfyUI-Hangover-Moondream
Apache License 2.0
37 stars 5 forks source link

Error occurred when executing Moondream Interrogator #20

Open niko2020 opened 3 days ago

niko2020 commented 3 days ago

Hi [Hangover3832],

Thanks for your last update. everything was perfectly working until today, I got the following error:

Error occurred when executing Moondream Interrogator:

index is out of bounds for dimension with size 0

File "/workspace/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/workspace/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/workspace/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) File "/workspace/ComfyUI/custom_nodes/ComfyUI-Hangover-Moondream/ho_moondream.py", line 155, in interrogate answer = self.model.answer_question(enc_image, p, self.tokenizer, temperature=temperature, do_sample=do_sample) File "/root/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/af5f98991c2661d0c4448ff6c2fb3c19e74f1c02/moondream.py", line 96, in answer_question answer = self.generate( File "/root/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/af5f98991c2661d0c4448ff6c2fb3c19e74f1c02/moondream.py", line 80, in generate output_ids = self.text_model.generate( File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "/workspace/venv_nextgen/lib/python3.10/site-packages/transformers/generation/utils.py", line 1914, in generate result = self._sample( File "/workspace/venv_nextgen/lib/python3.10/site-packages/transformers/generation/utils.py", line 2651, in _sample outputs = self( File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/af5f98991c2661d0c4448ff6c2fb3c19e74f1c02/modeling_phi.py", line 1074, in forward outputs = self.transformer( File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/af5f98991c2661d0c4448ff6c2fb3c19e74f1c02/modeling_phi.py", line 929, in forward layer_outputs = decoder_layer( File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/root/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/af5f98991c2661d0c4448ff6c2fb3c19e74f1c02/modeling_phi.py", line 733, in forward attn_outputs, self_attn_weights, present_key_value = self.mixer( File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/workspace/venv_nextgen/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/root/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/af5f98991c2661d0c4448ff6c2fb3c19e74f1c02/modeling_phi.py", line 382, in forward query_rot, key_rot = apply_rotary_pos_emb( File "/root/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/af5f98991c2661d0c4448ff6c2fb3c19e74f1c02/modeling_phi.py", line 214, in apply_rotary_pos_emb cos = cos[position_ids].unsqueeze(unsqueeze_dim)

Cheers

niko2020 commented 3 days ago

FYI it looks like it's related to a python package

niko2020 commented 3 days ago

transformers==4.41.2 required

Hangover3832 commented 2 days ago

@niko2020 Thank you, interesting, for me it works fine with transformers 4.39.3 and the latest model 2024-05-20. Do you remember from what version you updatet to 4.41.2?

niko2020 commented 2 days ago

What I meant was that transformers above 4.41.2 created problems.

Sorry, I should have said "transformers<=4.41.2 required"