search

Hangover3832 / ComfyUI-Hangover-Moondream

Moondream is a lightweight multimodal large language model
https://github.com/Hangover3832/ComfyUI-Hangover-Moondream
Apache License 2.0
37 stars 5 forks source link

Cannot import name `ToImage` error #9

Closed jquintanilla4 closed 2 months ago

jquintanilla4 commented 3 months ago

I got the following error when executing your node within a workflow. Comfy is updated to the latest, and have the following parameters.

image

Error occurred when executing Moondream Interrogator:

cannot import name 'ToImage' from 'torchvision.transforms.v2' (C:\sd\ComfyUI_windows_portable\python_embeded\lib\site-packages\torchvision\transforms\v2\__init__.py)

File "C:\sd\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\sd\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\sd\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\sd\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Hangover-Moondream\ho_moondream.py", line 56, in interrogate
self.model = AutoModelForCausalLM.from_pretrained(huggingface_model, trust_remote_code=trust_remote_code).to(dev)
File "C:\sd\ComfyUI_windows_portable\python_embeded\lib\site-packages\transformers\models\auto\auto_factory.py", line 548, in from_pretrained
model_class = get_class_from_dynamic_module(
File "C:\sd\ComfyUI_windows_portable\python_embeded\lib\site-packages\transformers\dynamic_module_utils.py", line 500, in get_class_from_dynamic_module
return get_class_in_module(class_name, final_module.replace(".py", ""))
File "C:\sd\ComfyUI_windows_portable\python_embeded\lib\site-packages\transformers\dynamic_module_utils.py", line 200, in get_class_in_module
module = importlib.import_module(module_path)
File "importlib\__init__.py", line 126, in import_module
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "C:\Users\dali\.cache\huggingface\modules\transformers_modules\vikhyatk\moondream2\0691a810b729384352a1bb0db09a43bb2d80e283\moondream.py", line 2, in
from .vision_encoder import VisionEncoder
File "C:\Users\dali\.cache\huggingface\modules\transformers_modules\vikhyatk\moondream2\0691a810b729384352a1bb0db09a43bb2d80e283\vision_encoder.py", line 5, in
from torchvision.transforms.v2 import (
Hangover3832 commented 3 months ago

@jquintanilla4 can you please run C:\sd\ComfyUI_windows_portable\python_embedded\python.exe -m pip freeze | findstr "timm einops transformers" and post the output.

jquintanilla4 commented 3 months ago

@Hangover3832 

einops @ file:///D:/a/ComfyUI/cu118_python_deps/einops-0.6.1-py3-none-any.whl#sha256=99149e46cc808956b174932fe563d920db4d6e5dadb8c6ecdaa7483b7ef7cfc3
timm==0.6.13
transformers==4.38.2
Hangover3832 commented 3 months ago

timm needs an upgrade, einops probably too. You can do this with C:\sd\ComfyUI_windows_portable\python_embedded\python.exe -m pip install timm einops --upgrade

jquintanilla4 commented 3 months ago

Just updated both. Still the same issue:

Error occurred when executing Moondream Interrogator:

cannot import name 'ToImage' from 'torchvision.transforms.v2' (C:\sd\ComfyUI_windows_portable\python_embeded\lib\site-packages\torchvision\transforms\v2\__init__.py)

File "C:\sd\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\sd\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\sd\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\sd\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Hangover-Moondream\ho_moondream.py", line 56, in interrogate
self.model = AutoModelForCausalLM.from_pretrained(huggingface_model, trust_remote_code=trust_remote_code).to(dev)
File "C:\sd\ComfyUI_windows_portable\python_embeded\lib\site-packages\transformers\models\auto\auto_factory.py", line 548, in from_pretrained
model_class = get_class_from_dynamic_module(
File "C:\sd\ComfyUI_windows_portable\python_embeded\lib\site-packages\transformers\dynamic_module_utils.py", line 500, in get_class_from_dynamic_module
return get_class_in_module(class_name, final_module.replace(".py", ""))
File "C:\sd\ComfyUI_windows_portable\python_embeded\lib\site-packages\transformers\dynamic_module_utils.py", line 200, in get_class_in_module
module = importlib.import_module(module_path)
File "importlib\__init__.py", line 126, in import_module
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "C:\Users\dali\.cache\huggingface\modules\transformers_modules\vikhyatk\moondream2\0691a810b729384352a1bb0db09a43bb2d80e283\moondream.py", line 2, in
from .vision_encoder import VisionEncoder
File "C:\Users\dali\.cache\huggingface\modules\transformers_modules\vikhyatk\moondream2\0691a810b729384352a1bb0db09a43bb2d80e283\vision_encoder.py", line 5, in
from torchvision.transforms.v2 import (

For reference, after update:

einops==0.7.0
timm==0.9.16
transformers==4.38.2
Hangover3832 commented 3 months ago

What about C:\sd\ComfyUI_windows_portable\python_embedded\python.exe -m pip freeze | findstr "torch"

jquintanilla4 commented 3 months ago 
open-clip-torch==2.24.0
torch @ file:///D:/a/ComfyUI/cu118_python_deps/torch-2.0.1%2Bcu118-cp310-cp310-win_amd64.whl#sha256=f58d75619bc96e4322343c030b893613701caa2d6db8017155da226c14171335
torchaudio @ file:///D:/a/ComfyUI/cu118_python_deps/torchaudio-2.0.2%2Bcu118-cp310-cp310-win_amd64.whl#sha256=8ad0e93e39ce594a5cc53d46d972fde20b499e69d1e7f765adb33de13cfbab86
torchsde @ file:///D:/a/ComfyUI/cu118_python_deps/torchsde-0.2.5-py3-none-any.whl#sha256=4c34373a94a357bdf60bbfee00c850f3563d634491555820b900c9a4f7eff300
torchvision @ file:///D:/a/ComfyUI/cu118_python_deps/torchvision-0.15.2%2Bcu118-cp310-cp310-win_amd64.whl#sha256=bfd2435d681418bea8dacde2b2cb6e5dd40a0e0243d3631e2b71c10bf9831f39
Hangover3832 commented 3 months ago

@jquintanilla4, this might be the problem, mine is:

torch @ file:///D:/a/ComfyUI/cu121_python_deps/torch-2.2.0%2Bcu121-cp311-cp311-win_amd64.whl#sha256=d79324159c622243429ec214a86b8613c1d7d46fc4821374d324800f1df6ade1
torchaudio @ file:///D:/a/ComfyUI/cu121_python_deps/torchaudio-2.2.0%2Bcu121-cp311-cp311-win_amd64.whl#sha256=67a33d2066668a2754d9dd5d000419f60102dd17eff9803f9b0a5e1d9261f79d
torchsde @ file:///D:/a/ComfyUI/cu121_python_deps/torchsde-0.2.6-py3-none-any.whl#sha256=19bf7ff02eec7e8e46ba1cdb4aa0f9db1c51d492524a16975234b467f7fc463b
torchvision @ file:///D:/a/ComfyUI/cu121_python_deps/torchvision-0.17.0%2Bcu121-cp311-cp311-win_amd64.whl#sha256=307e52c2887c1d2b50cc3581cf5f4c169130b8352462e361e71eeda19e0dd263

Your installation seems quite outdated, but I'm really not 100% sure about this.

jquintanilla4 commented 3 months ago

Yeah I keep my CUDA Toolkit at 11.8 because it's a happy medium for older versions of PyTorch and newer. I need that flexibility for ML research.

Well no worries. When I find some time I'll dig into making Moondream work with 11.8 and earlier versions of PyTorch 2, I'll ping this issue with an update. Thanks for your help.