WAS Node Suite Error: `transformers==4.26.1` is required for BLIP models

[kakochka]

When trying to execute Blip Analyze Image, an error came out:

_WAS Node Suite Error: transformers==4.26.1 is required for BLIP models. Please run the C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\requirements.txt through ComfyUI's ptyhon executable. reshape position embedding from 196 to 576 load checkpoint from https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_capfilt_large.pth ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "C:\Users\kakochka\ComfyUI\ComfyUI\execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\execution.py", line 83, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\execution.py", line 76, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 10866, in blip_caption_image caption = model.generate(tensor, sample=False, num_beams=6, max_length=74, min_length=20) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_module.py", line 162, in generate outputs = self.text_decoder.generate(input_ids=input_ids, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\transformers\generation\utils.py", line 1752, in generate return self.beam_search( ^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\transformers\generation\utils.py", line 3091, in beam_search outputs = self( ^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_med.py", line 886, in forward outputs = self.bert( ^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_med.py", line 781, in forward encoder_outputs = self.encoder( ^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_med.py", line 445, in forward layer_outputs = layer_module( ^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_med.py", line 361, in forward cross_attention_outputs = self.crossattention( ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_med.py", line 277, in forward self_outputs = self.self( ^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_med.py", line 178, in forward attention_scores = torch.matmul(query_layer, keylayer.transpose(-1, -2)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: The size of tensor a (6) must match the size of tensor b (36) at non-singleton dimension 0

I've tried to install requirements.txt. (cmd from the was-node-suite-comfyui and pip install -r requirements.txt). Everything went fine, but after restarting Comfy nothing's changed. I've reinstalled was-node-suite-comfyui but that didn't helped either - the error's still there. I'm not sure what "Please run the C:\Users\kakochka\ComfyUI\ComfyUI\custom_nodes\was-node-suite-comfyui\requirements.txt through ComfyUI's ptyhon executable." means, so maybe I've installed requirements somehow wrong. Pls help!

[angrysky56]

This worked- just double click.

[WASasquatch]

Yeah that error is from newer transformers that seems to have abandoned blip and the documentation isn't covering changes.

On Wed, Dec 6, 2023, 9:32 PM angrysky56 @.***> wrote:

image.png (view on web) https://github.com/WASasquatch/was-node-suite-comfyui/assets/113643118/c96f2043-8af0-4552-9b30-084162747af9 This worked- just double click.

— Reply to this email directly, view it on GitHub https://github.com/WASasquatch/was-node-suite-comfyui/issues/293#issuecomment-1844511545, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIZEZKPAVHBPFCZDXLVCHTYIFIG5AVCNFSM6AAAAABAE26HHOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBUGUYTCNJUGU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Latentnaut]

not working for me...

`ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "urllib\request.py", line 1348, in do_open File "http\client.py", line 1286, in request File "http\client.py", line 1332, in _send_request File "http\client.py", line 1281, in endheaders File "http\client.py", line 1041, in _send_output File "http\client.py", line 979, in send File "http\client.py", line 1458, in connect File "ssl.py", line 517, in wrap_socket File "ssl.py", line 1108, in _create File "ssl.py", line 1379, in do_handshake ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 10824, in blip_model model = blip_vqa(pretrained=model_url, image_size=size, vit='base') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_module.py", line 410, in blip_vqa model,msg = load_checkpoint(model,pretrained) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_module.py", line 223, in load_checkpoint cached_file = download_cached_file(url_or_filename, check_hash=False, progress=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\timm\models\hub.py", line 58, in download_cached_file download_url_to_file(url, cached_file, hash_prefix, progress=progress) File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\hub.py", line 620, in download_url_to_file u = urlopen(req) ^^^^^^^^^^^^ File "urllib\request.py", line 216, in urlopen File "urllib\request.py", line 519, in open File "urllib\request.py", line 536, in _open File "urllib\request.py", line 496, in _call_chain File "urllib\request.py", line 1391, in https_open File "urllib\request.py", line 1351, in do_open urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)>`

[Latentnaut]

Same here.

[wsry888]

how to fix this?

[Latentnaut]

Hey, I forgot BLIP, now I am happy with image2prompt: https://github.com/zhongpei/Comfyui_image2prompt

[WASasquatch]

not working for me...

`ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "urllib\request.py", line 1348, in do_open File "http\client.py", line 1286, in request File "http\client.py", line 1332, in _send_request File "http\client.py", line 1281, in endheaders File "http\client.py", line 1041, in _send_output File "http\client.py", line 979, in send File "http\client.py", line 1458, in connect File "ssl.py", line 517, in wrap_socket File "ssl.py", line 1108, in _create File "ssl.py", line 1379, in do_handshake ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 10824, in blip_model model = blip_vqa(pretrained=model_url, image_size=size, vit='base') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_module.py", line 410, in blip_vqa model,msg = load_checkpoint(model,pretrained) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\modules\BLIP\blip_module.py", line 223, in load_checkpoint cached_file = download_cached_file(url_or_filename, check_hash=False, progress=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\timm\models\hub.py", line 58, in download_cached_file download_url_to_file(url, cached_file, hash_prefix, progress=progress) File "C:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\hub.py", line 620, in download_url_to_file u = urlopen(req) ^^^^^^^^^^^^ File "urllib\request.py", line 216, in urlopen File "urllib\request.py", line 519, in open File "urllib\request.py", line 536, in _open File "urllib\request.py", line 496, in _call_chain File "urllib\request.py", line 1391, in https_open File "urllib\request.py", line 1351, in do_open urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)>`

This isn't even a transformers error. Your machine has likely outdated TLS and not supporting new SSL secure protocol. This is a good error to protect you.

[Latentnaut]

I am not sure what does it means, my machine is relative new and it has a fresh new Windows 11 installation. Does it make sense or I don't have a clue about what's a outdated TLS?

[WAS-PlaiLabs]

BLIP has been refactored

https://github.com/WASasquatch/was-node-suite-comfyui/pull/393