search

zhongpei / Comfyui_image2prompt

image to prompt by vikhyatk/moondream1
GNU General Public License v3.0
243 stars 14 forks source link

NotImplementedError: Make sure `_init_weights` is implemented #8

Closed Wraithnaut closed 3 months ago

Wraithnaut commented 5 months ago 
         It may be a version problem,you can try
pip install transformers==4.37.2

_Originally posted by @zhongpei in https://github.com/zhongpei/Comfyui_image2prompt/issues/1#issuecomment-1918338658_

I had an issue with init weights but updating transformers from 4.26.1 to 4.37.2 appears to have fixed the issue.

ERROR:root:!!! Exception during processing !!!
[2024-02-02 20:16] ERROR:root:Traceback (most recent call last):
  File "/home/USERNAME/ComfyUI/execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/USERNAME/ComfyUI/execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/USERNAME/ComfyUI/execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/USERNAME/ComfyUI/custom_nodes/Comfyui_image2prompt/image2text.py", line 29, in get_model
    return (MoondreamModel(device=device,low_memory=low_memory),)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/USERNAME/ComfyUI/custom_nodes/Comfyui_image2prompt/moondream_model.py", line 1177, in __init__
    self.model_path, self.vision_encoder, self.text_model = self.load_model(device=device, dtype=dtype)
                                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/USERNAME/ComfyUI/custom_nodes/Comfyui_image2prompt/moondream_model.py", line 1196, in load_model
    text_model = TextModel(model_path).to(device=device, dtype=dtype)
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/home/USERNAME/ComfyUI/custom_nodes/Comfyui_image2prompt/moondream_model.py", line 1084, in __init__
    self.model = PhiForCausalLM(phi_config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/USERNAME/ComfyUI/custom_nodes/Comfyui_image2prompt/moondream_model.py", line 1012, in __init__
    self.transformer = PhiModel(config)
                       ^^^^^^^^^^^^^^^^
  File "/home/USERNAME/ComfyUI/custom_nodes/Comfyui_image2prompt/moondream_model.py", line 954, in __init__
    self.post_init()
  File "/home/USERNAME/ComfyUI/venv/lib64/python3.11/site-packages/transformers/modeling_utils.py", line 1043, in post_init
    self.init_weights()
  File "/home/USERNAME/ComfyUI/venv/lib64/python3.11/site-packages/transformers/modeling_utils.py", line 1507, in init_weights
    self.apply(self._init_weights)
  File "/home/USERNAME/ComfyUI/venv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 897, in apply
    module.apply(fn)
  File "/home/USERNAME/ComfyUI/venv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 897, in apply
    module.apply(fn)
  File "/home/USERNAME/ComfyUI/venv/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 898, in apply
    fn(self)
  File "/home/USERNAME/ComfyUI/venv/lib64/python3.11/site-packages/transformers/modeling_utils.py", line 1170, in _init_weights
    raise NotImplementedError(f"Make sure `_init_weights` is implemented for {self.__class__}")
NotImplementedError: Make sure `_init_weights` is implemented for <class 'Comfyui_image2prompt.moondream_model.PhiModel'>
zhongpei commented 3 months ago

pip install transformers==4.37.1