search

kijai / ComfyUI-moondream

ComfyUI node to use the moondream tiny vision language model
80 stars 8 forks source link

new weights shape #15

Open rvorias opened 1 month ago

rvorias commented 1 month ago

https://github.com/kijai/ComfyUI-moondream/blob/b97ad4718821d7cee5eacce139c94c9de51268b8/moondream/vision_encoder.py#L37

new revision might have a different shape: 2304

Traceback (most recent call last):
  File "/root/ComfyUI/execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "/root/ComfyUI/execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "/root/ComfyUI/execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "/root/ComfyUI/custom_nodes/ComfyUI-moondream/nodes.py", line 62, in process
    self.moondream = Moondream.from_pretrained(checkpoint_path).to(device=device, dtype=dtype)
  File "/usr/local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3850, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/usr/local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4335, in _load_pretrained_model
    raise RuntimeError(f"Error(s) in loading state_dict for {model.__class__.__name__}:\n\t{error_msg}")
RuntimeError: Error(s) in loading state_dict for Moondream:
    size mismatch for vision_encoder.projection.mlp.fc1.weight: copying a param with shape torch.Size([8192, 2304]) from checkpoint, the shape in current model is torch.Size([8192, 1152]).
    You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.
niko2020 commented 4 weeks ago

same here