search

EvilBT / ComfyUI_SLK_joy_caption_two

ComfyUI Node
GNU General Public License v3.0
192 stars 6 forks source link

error message : "Input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same....... #31

Open AALeslie2468 opened 1 month ago

AALeslie2468 commented 1 month ago

after click "queue Prompt", then the error message :

Input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

VAVAVAAA commented 1 month ago

I'm getting the same error

EvilBT commented 1 month ago

Please post the complete error message.

EvilBT commented 1 month ago

1

AALeslie2468 commented 1 month ago

Thanks. The message in "comfyui.log" :

ComfyUI-Manager: installing dependencies done.

[2024-10-18 14:27] ComfyUI startup time: 2024-10-18 14:27:01.555321 [2024-10-18 14:27] Platform: Windows [2024-10-18 14:27] Python version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)] [2024-10-18 14:27] Python executable: D:\AI\ComfyUI_windows_portable\python_embeded\python.exe [2024-10-18 14:27] ComfyUI Path: D:\AI\ComfyUI_windows_portable\ComfyUI [2024-10-18 14:27] Log path: D:\AI\ComfyUI_windows_portable\comfyui.log [2024-10-18 14:27] Prestartup times for custom nodes: [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use [2024-10-18 14:27] 2.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager [2024-10-18 14:27] Total VRAM 6144 MB, total RAM 32539 MB [2024-10-18 14:27] pytorch version: 2.2.2+cu121 [2024-10-18 14:27] Set vram state to: NORMAL_VRAM [2024-10-18 14:27] Device: cuda:0 NVIDIA GeForce GTX 1660 SUPER : cudaMallocAsync [2024-10-18 14:27] Using pytorch cross attention [2024-10-18 14:27] [Prompt Server] web root: D:\AI\ComfyUI_windows_portable\ComfyUI\web [2024-10-18 14:27] Adding extra search path checkpoints D:\AI\stable-diffusion-webui\models/Stable-diffusion [2024-10-18 14:27] Adding extra search path configs D:\AI\stable-diffusion-webui\models/Stable-diffusion [2024-10-18 14:27] Adding extra search path vae D:\AI\stable-diffusion-webui\models/VAE [2024-10-18 14:27] Adding extra search path loras D:\AI\stable-diffusion-webui\models/Lora [2024-10-18 14:27] Adding extra search path loras D:\AI\stable-diffusion-webui\models/LyCORIS [2024-10-18 14:27] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/ESRGAN [2024-10-18 14:27] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/RealESRGAN [2024-10-18 14:27] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/SwinIR [2024-10-18 14:27] Adding extra search path embeddings D:\AI\stable-diffusion-webui\embeddings [2024-10-18 14:27] Adding extra search path hypernetworks D:\AI\stable-diffusion-webui\models/hypernetworks [2024-10-18 14:27] Adding extra search path controlnet D:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\models [2024-10-18 14:27] [Crystools INFO] Crystools version: 1.19.0 [2024-10-18 14:27] [Crystools INFO] CPU: 13th Gen Intel(R) Core(TM) i5-13500 - Arch: AMD64 - OS: Windows 10 [2024-10-18 14:27] [Crystools INFO] Pynvml (Nvidia) initialized. [2024-10-18 14:27] [Crystools INFO] GPU/s: [2024-10-18 14:27] [Crystools INFO] 0) NVIDIA GeForce GTX 1660 SUPER [2024-10-18 14:27] [Crystools INFO] NVIDIA Driver: 560.94 [2024-10-18 14:27] [ComfyUI-Easy-Use] server: v1.2.4 Loaded [2024-10-18 14:27] [ComfyUI-Easy-Use] web root: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use\web_version/v2 Loaded [2024-10-18 14:27] ### Loading: ComfyUI-Impact-Pack (V7.10.3) [2024-10-18 14:27] ### Loading: ComfyUI-Impact-Pack (Subpack: V0.7) [2024-10-18 14:27] [Impact Pack] Wildcards loading done. [2024-10-18 14:27] ### Loading: ComfyUI-Inspire-Pack (V1.5.1) [2024-10-18 14:27] ### Loading: ComfyUI-Manager (V2.51.8) [2024-10-18 14:27] ### ComfyUI Revision: 2770 [ce0542e1] | Released on '2024-10-17' [2024-10-18 14:27] [SD Prompt Reader] Node version: 1.3.4 [2024-10-18 14:27] [SD Prompt Reader] Core version: 1.3.5 [2024-10-18 14:27] ------------------------------------------ [2024-10-18 14:27] Comfyroll Studio v1.76 :  175 Nodes Loaded [2024-10-18 14:27] ------------------------------------------ [2024-10-18 14:27] For changes, please see patch notes at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/blob/main/Patch_Notes.md [2024-10-18 14:27] For help, please see the wiki at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/wiki [2024-10-18 14:27] ------------------------------------------ [2024-10-18 14:27] [comfyui_controlnet_aux] | INFO -> Using ckpts path: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts [2024-10-18 14:27] [comfyui_controlnet_aux] | INFO -> Using symlinks: False [2024-10-18 14:27] [comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider'] [2024-10-18 14:27] D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\dwpose.py:26: UserWarning: DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly warnings.warn("DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly") [2024-10-18 14:27] Efficiency Nodes: Attempting to add Control Net options to the 'HiRes-Fix Script' Node (comfyui_controlnet_aux add-on)...Success! [2024-10-18 14:27] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json [2024-10-18 14:27] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json [2024-10-18 14:27] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json [2024-10-18 14:27] [2024-10-18 14:27] [rgthree-comfy] Loaded 42 exciting nodes. 🎉 [2024-10-18 14:27] [2024-10-18 14:27] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [2024-10-18 14:27] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json [2024-10-18 14:27] WAS Node Suite: OpenCV Python FFMPEG support is enabled [2024-10-18 14:27] WAS Node Suite Warning: ffmpeg_bin_path is not set in D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\was_suite_config.json config file. Will attempt to use system ffmpeg binaries if available. [2024-10-18 14:27] WAS Node Suite: Finished. Loaded 218 nodes successfully. [2024-10-18 14:27] "Every strike brings me closer to the next home run." - Babe Ruth [2024-10-18 14:27] [2024-10-18 14:27] Import times for custom nodes: [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\websocket_image_save.py [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Noise [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-GGUF [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Derfuu_ComfyUI_ModdedNodes [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-prompt-reader-node [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack [2024-10-18 14:27] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_bitsandbytes_NF4 [2024-10-18 14:27] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux [2024-10-18 14:27] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes [2024-10-18 14:27] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Crystools [2024-10-18 14:27] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\facerestore_cf [2024-10-18 14:27] 0.3 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager [2024-10-18 14:27] 0.4 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack [2024-10-18 14:27] 0.7 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use [2024-10-18 14:27] 2.2 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui [2024-10-18 14:27] [2024-10-18 14:27] Starting server

[2024-10-18 14:27] To see the GUI go to: http://127.0.0.1:8188 [2024-10-18 14:27] FETCH DATA from: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE] [2024-10-18 14:27] [Inspire Pack] IPAdapterPlus is not installed. [2024-10-18 14:27] got prompt [2024-10-18 14:27] D:\AI\ComfyUI_windows_portable\ComfyUI\models\clip\siglip-so400m-patch14-384 [2024-10-18 14:27] Loading VLM's custom vision model [2024-10-18 14:27] Requested to load SiglipVisionTransformer [2024-10-18 14:27] Loading 1 new model [2024-10-18 14:27] loaded completely 0.0 809.1729736328125 True [2024-10-18 14:27] !!! Exception during processing !!! Input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor [2024-10-18 14:27] Traceback (most recent call last): File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two\joy_caption_two_node.py", line 332, in generate vision_outputs = joy_two_pipeline.clip_model.encode_image(pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two\joy_caption_two_node.py", line 67, in encode_image vision_outputs = self.model(pixel_values=pixel_values, output_hidden_states=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 1087, in forward hidden_states = self.embeddings(pixel_values, interpolate_pos_encoding=interpolate_pos_encoding) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 310, in forward patch_embeds = self.patch_embedding(pixel_values) # shape = [, width, grid, grid] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\conv.py", line 460, in forward return self._conv_forward(input, self.weight, self.bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\conv.py", line 456, in _conv_forward return F.conv2d(input, weight, bias, self.stride, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

[2024-10-18 14:27] Prompt executed in 2.13 seconds

Thanks

EvilBT commented 1 month ago

Could you please test it for me first? I'll hold off on changing the code for now. In joy_caption_two_node.py, around line 132, add these two lines of code:

        image_adapter.load_state_dict(torch.load(adapter_path, map_location=self.offload_device, weights_only=True))
        # New added lines
        img_dtype = text_encoder_dtype()
        image_adapter = image_adapter.to(img_dtype)
        # This is to specify the conversion to a unified format
        image_adapter.eval()

Insert these two lines between the existing load_state_dict and eval() calls. This should ensure the image_adapter is loaded in the same precision as other parts of the model.

After adding these lines, please restart ComfyUI and test it to see if the following error is resolved:

Exception Message: Input type (torch.cuda.HalfTensor) and weight type (torch.FloatTensor) should be the same

Please let me know the test results so I can modify the code accordingly.

cchance27 commented 1 month ago

wouldn't it be better to do

pixel_values = pixel_values.to(dtype=self.type) on the image_encode to force the image down to the model's dtype?

AALeslie2468 commented 1 month ago 
img_dtype = text_encoder_dtype()

unfortunately, still the same error massage:

the massage in "comfyui.log" :

ComfyUI-Manager: installing dependencies done.

[2024-10-19 15:20] ComfyUI startup time: 2024-10-19 15:20:18.493789 [2024-10-19 15:20] Platform: Windows [2024-10-19 15:20] Python version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)] [2024-10-19 15:20] Python executable: D:\AI\ComfyUI_windows_portable\python_embeded\python.exe [2024-10-19 15:20] ComfyUI Path: D:\AI\ComfyUI_windows_portable\ComfyUI [2024-10-19 15:20] Log path: D:\AI\ComfyUI_windows_portable\comfyui.log [2024-10-19 15:20] Prestartup times for custom nodes: [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use [2024-10-19 15:20] 2.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager [2024-10-19 15:20] Total VRAM 6144 MB, total RAM 32539 MB [2024-10-19 15:20] pytorch version: 2.2.2+cu121 [2024-10-19 15:20] Set vram state to: NORMAL_VRAM [2024-10-19 15:20] Device: cuda:0 NVIDIA GeForce GTX 1660 SUPER : cudaMallocAsync [2024-10-19 15:20] Using pytorch cross attention [2024-10-19 15:20] [Prompt Server] web root: D:\AI\ComfyUI_windows_portable\ComfyUI\web [2024-10-19 15:20] Adding extra search path checkpoints D:\AI\stable-diffusion-webui\models/Stable-diffusion [2024-10-19 15:20] Adding extra search path configs D:\AI\stable-diffusion-webui\models/Stable-diffusion [2024-10-19 15:20] Adding extra search path vae D:\AI\stable-diffusion-webui\models/VAE [2024-10-19 15:20] Adding extra search path loras D:\AI\stable-diffusion-webui\models/Lora [2024-10-19 15:20] Adding extra search path loras D:\AI\stable-diffusion-webui\models/LyCORIS [2024-10-19 15:20] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/ESRGAN [2024-10-19 15:20] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/RealESRGAN [2024-10-19 15:20] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/SwinIR [2024-10-19 15:20] Adding extra search path embeddings D:\AI\stable-diffusion-webui\embeddings [2024-10-19 15:20] Adding extra search path hypernetworks D:\AI\stable-diffusion-webui\models/hypernetworks [2024-10-19 15:20] Adding extra search path controlnet D:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\models [2024-10-19 15:20] [Crystools INFO] Crystools version: 1.19.0 [2024-10-19 15:20] [Crystools INFO] CPU: 13th Gen Intel(R) Core(TM) i5-13500 - Arch: AMD64 - OS: Windows 10 [2024-10-19 15:20] [Crystools INFO] Pynvml (Nvidia) initialized. [2024-10-19 15:20] [Crystools INFO] GPU/s: [2024-10-19 15:20] [Crystools INFO] 0) NVIDIA GeForce GTX 1660 SUPER [2024-10-19 15:20] [Crystools INFO] NVIDIA Driver: 560.94 [2024-10-19 15:20] [ComfyUI-Easy-Use] server: v1.2.4 Loaded [2024-10-19 15:20] [ComfyUI-Easy-Use] web root: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use\web_version/v2 Loaded [2024-10-19 15:20] ### Loading: ComfyUI-Impact-Pack (V7.10.3) [2024-10-19 15:20] ### Loading: ComfyUI-Impact-Pack (Subpack: V0.7) [2024-10-19 15:20] [Impact Pack] Wildcards loading done. [2024-10-19 15:20] ### Loading: ComfyUI-Inspire-Pack (V1.5.1) [2024-10-19 15:20] ### Loading: ComfyUI-Manager (V2.51.8) [2024-10-19 15:20] ### ComfyUI Revision: 2771 [518c0dc2] | Released on '2024-10-18' [2024-10-19 15:20] [SD Prompt Reader] Node version: 1.3.4 [2024-10-19 15:20] [SD Prompt Reader] Core version: 1.3.5 [2024-10-19 15:20] ------------------------------------------ [2024-10-19 15:20] Comfyroll Studio v1.76 :  175 Nodes Loaded [2024-10-19 15:20] ------------------------------------------ [2024-10-19 15:20] For changes, please see patch notes at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/blob/main/Patch_Notes.md [2024-10-19 15:20] For help, please see the wiki at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/wiki [2024-10-19 15:20] ------------------------------------------ [2024-10-19 15:20] [comfyui_controlnet_aux] | INFO -> Using ckpts path: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts [2024-10-19 15:20] [comfyui_controlnet_aux] | INFO -> Using symlinks: False [2024-10-19 15:20] [comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider'] [2024-10-19 15:20] D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\dwpose.py:26: UserWarning: DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly warnings.warn("DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly") [2024-10-19 15:20] Efficiency Nodes: Attempting to add Control Net options to the 'HiRes-Fix Script' Node (comfyui_controlnet_aux add-on)...Success! [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json [2024-10-19 15:20] [2024-10-19 15:20] [rgthree-comfy] Loaded 42 epic nodes. 🎉 [2024-10-19 15:20] [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [2024-10-19 15:20] WAS Node Suite: OpenCV Python FFMPEG support is enabled [2024-10-19 15:20] WAS Node Suite Warning: ffmpeg_bin_path is not set in D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\was_suite_config.json config file. Will attempt to use system ffmpeg binaries if available. [2024-10-19 15:20] WAS Node Suite: Finished. Loaded 218 nodes successfully. [2024-10-19 15:20] "Be the change that you wish to see in the world." - Mahatma Gandhi [2024-10-19 15:20] [2024-10-19 15:20] Import times for custom nodes: [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\websocket_image_save.py [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Noise [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-GGUF [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Derfuu_ComfyUI_ModdedNodes [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-prompt-reader-node [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_bitsandbytes_NF4 [2024-10-19 15:20] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux [2024-10-19 15:20] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes [2024-10-19 15:20] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Crystools [2024-10-19 15:20] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\facerestore_cf [2024-10-19 15:20] 0.3 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager [2024-10-19 15:20] 0.4 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack [2024-10-19 15:20] 0.6 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use [2024-10-19 15:20] 2.3 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui [2024-10-19 15:20] [2024-10-19 15:20] Starting server

[2024-10-19 15:20] To see the GUI go to: http://127.0.0.1:8188 [2024-10-19 15:20] FETCH DATA from: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE] [2024-10-19 15:20] [Inspire Pack] IPAdapterPlus is not installed. [2024-10-19 15:20] got prompt [2024-10-19 15:20] D:\AI\ComfyUI_windows_portable\ComfyUI\models\clip\siglip-so400m-patch14-384 [2024-10-19 15:20] Loading VLM's custom vision model [2024-10-19 15:20] Requested to load SiglipVisionTransformer [2024-10-19 15:20] Loading 1 new model [2024-10-19 15:20] loaded completely 0.0 809.1729736328125 True [2024-10-19 15:20] !!! Exception during processing !!! Input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor [2024-10-19 15:20] Traceback (most recent call last): File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two\joy_caption_two_node.py", line 341, in generate vision_outputs = joy_two_pipeline.clip_model.encode_image(pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two\joy_caption_two_node.py", line 68, in encode_image vision_outputs = self.model(pixel_values=pixel_values, output_hidden_states=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 1087, in forward hidden_states = self.embeddings(pixel_values, interpolate_pos_encoding=interpolate_pos_encoding) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 310, in forward patch_embeds = self.patch_embedding(pixel_values) # shape = [, width, grid, grid] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\conv.py", line 460, in forward return self._conv_forward(input, self.weight, self.bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\conv.py", line 456, in _conv_forward return F.conv2d(input, weight, bias, self.stride, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

[2024-10-19 15:20] Prompt executed in 2.21 seconds

Thanks

EvilBT commented 1 month ago 
img_dtype = text_encoder_dtype()

unfortunately, still the same error massage:

the massage in "comfyui.log" :

ComfyUI-Manager: installing dependencies done.

[2024-10-19 15:20] ComfyUI startup time: 2024-10-19 15:20:18.493789 [2024-10-19 15:20] Platform: Windows [2024-10-19 15:20] Python version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)] [2024-10-19 15:20] Python executable: D:\AI\ComfyUI_windows_portable\python_embeded\python.exe [2024-10-19 15:20] ComfyUI Path: D:\AI\ComfyUI_windows_portable\ComfyUI [2024-10-19 15:20] Log path: D:\AI\ComfyUI_windows_portable\comfyui.log [2024-10-19 15:20] Prestartup times for custom nodes: [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use [2024-10-19 15:20] 2.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager [2024-10-19 15:20] Total VRAM 6144 MB, total RAM 32539 MB [2024-10-19 15:20] pytorch version: 2.2.2+cu121 [2024-10-19 15:20] Set vram state to: NORMAL_VRAM [2024-10-19 15:20] Device: cuda:0 NVIDIA GeForce GTX 1660 SUPER : cudaMallocAsync [2024-10-19 15:20] Using pytorch cross attention [2024-10-19 15:20] [Prompt Server] web root: D:\AI\ComfyUI_windows_portable\ComfyUI\web [2024-10-19 15:20] Adding extra search path checkpoints D:\AI\stable-diffusion-webui\models/Stable-diffusion [2024-10-19 15:20] Adding extra search path configs D:\AI\stable-diffusion-webui\models/Stable-diffusion [2024-10-19 15:20] Adding extra search path vae D:\AI\stable-diffusion-webui\models/VAE [2024-10-19 15:20] Adding extra search path loras D:\AI\stable-diffusion-webui\models/Lora [2024-10-19 15:20] Adding extra search path loras D:\AI\stable-diffusion-webui\models/LyCORIS [2024-10-19 15:20] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/ESRGAN [2024-10-19 15:20] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/RealESRGAN [2024-10-19 15:20] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/SwinIR [2024-10-19 15:20] Adding extra search path embeddings D:\AI\stable-diffusion-webui\embeddings [2024-10-19 15:20] Adding extra search path hypernetworks D:\AI\stable-diffusion-webui\models/hypernetworks [2024-10-19 15:20] Adding extra search path controlnet D:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\models [2024-10-19 15:20] [Crystools �[0;32mINFO�[0m] Crystools version: 1.19.0 [2024-10-19 15:20] [Crystools �[0;32mINFO�[0m] CPU: 13th Gen Intel(R) Core(TM) i5-13500 - Arch: AMD64 - OS: Windows 10 [2024-10-19 15:20] [Crystools �[0;32mINFO�[0m] Pynvml (Nvidia) initialized. [2024-10-19 15:20] [Crystools �[0;32mINFO�[0m] GPU/s: [2024-10-19 15:20] [Crystools �[0;32mINFO�[0m] 0) NVIDIA GeForce GTX 1660 SUPER [2024-10-19 15:20] [Crystools �[0;32mINFO�[0m] NVIDIA Driver: 560.94 [2024-10-19 15:20] �[34m[ComfyUI-Easy-Use] server: �[0mv1.2.4 �[92mLoaded�[0m [2024-10-19 15:20] �[34m[ComfyUI-Easy-Use] web root: �[0mD:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use\web_version/v2 �[92mLoaded�[0m [2024-10-19 15:20] ### Loading: ComfyUI-Impact-Pack (V7.10.3) [2024-10-19 15:20] ### Loading: ComfyUI-Impact-Pack (Subpack: V0.7) [2024-10-19 15:20] [Impact Pack] Wildcards loading done. [2024-10-19 15:20] ### Loading: ComfyUI-Inspire-Pack (V1.5.1) [2024-10-19 15:20] ### Loading: ComfyUI-Manager (V2.51.8) [2024-10-19 15:20] ### ComfyUI Revision: 2771 [518c0dc2] | Released on '2024-10-18' [2024-10-19 15:20] �[0m�[1;34m[SD Prompt Reader] �[36mNode version: 1.3.4�[0m [2024-10-19 15:20] �[0m�[1;34m[SD Prompt Reader] �[36mCore version: 1.3.5�[0m [2024-10-19 15:20] ------------------------------------------ [2024-10-19 15:20] �[34mComfyroll Studio v1.76 : �[92m 175 Nodes Loaded�[0m [2024-10-19 15:20] ------------------------------------------ [2024-10-19 15:20] For changes, please see patch notes at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/blob/main/Patch_Notes.md [2024-10-19 15:20] For help, please see the wiki at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/wiki [2024-10-19 15:20] ------------------------------------------ [2024-10-19 15:20] �[36;20m[comfyui_controlnet_aux] | INFO -> Using ckpts path: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts�[0m [2024-10-19 15:20] �[36;20m[comfyui_controlnet_aux] | INFO -> Using symlinks: False�[0m [2024-10-19 15:20] �[36;20m[comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider']�[0m [2024-10-19 15:20] D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\dwpose.py:26: UserWarning: DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly warnings.warn("DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly") [2024-10-19 15:20] �[36mEfficiency Nodes:�[0m Attempting to add Control Net options to the 'HiRes-Fix Script' Node (comfyui_controlnet_aux add-on)...�[92mSuccess!�[0m [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json [2024-10-19 15:20] [2024-10-19 15:20] �[92m[rgthree-comfy] Loaded 42 epic nodes. 🎉�[00m [2024-10-19 15:20] [2024-10-19 15:20] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [2024-10-19 15:20] �[34mWAS Node Suite: �[0mOpenCV Python FFMPEG support is enabled�[0m [2024-10-19 15:20] �[34mWAS Node Suite �[93mWarning: �[0mffmpeg_bin_path is not set in D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\was_suite_config.json config file. Will attempt to use system ffmpeg binaries if available.�[0m [2024-10-19 15:20] �[34mWAS Node Suite: �[0mFinished.�[0m �[32mLoaded�[0m �[0m218�[0m �[32mnodes successfully.�[0m [2024-10-19 15:20] �[3m�[93m"Be the change that you wish to see in the world."�[0m�[3m - Mahatma Gandhi�[0m [2024-10-19 15:20] [2024-10-19 15:20] Import times for custom nodes: [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\websocket_image_save.py [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Noise [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-GGUF [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Derfuu_ComfyUI_ModdedNodes [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-prompt-reader-node [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack [2024-10-19 15:20] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_bitsandbytes_NF4 [2024-10-19 15:20] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux [2024-10-19 15:20] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes [2024-10-19 15:20] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Crystools [2024-10-19 15:20] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\facerestore_cf [2024-10-19 15:20] 0.3 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager [2024-10-19 15:20] 0.4 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack [2024-10-19 15:20] 0.6 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use [2024-10-19 15:20] 2.3 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui [2024-10-19 15:20] [2024-10-19 15:20] Starting server

[2024-10-19 15:20] To see the GUI go to: http://127.0.0.1:8188 [2024-10-19 15:20] FETCH DATA from: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE] [2024-10-19 15:20] [Inspire Pack] IPAdapterPlus is not installed. [2024-10-19 15:20] got prompt [2024-10-19 15:20] D:\AI\ComfyUI_windows_portable\ComfyUI\models\clip\siglip-so400m-patch14-384 [2024-10-19 15:20] Loading VLM's custom vision model [2024-10-19 15:20] Requested to load SiglipVisionTransformer [2024-10-19 15:20] Loading 1 new model [2024-10-19 15:20] loaded completely 0.0 809.1729736328125 True [2024-10-19 15:20] !!! Exception during processing !!! Input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor [2024-10-19 15:20] Traceback (most recent call last): File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two\joy_caption_two_node.py", line 341, in generate vision_outputs = joy_two_pipeline.clip_model.encode_image(pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two\joy_caption_two_node.py", line 68, in encode_image vision_outputs = self.model(pixel_values=pixel_values, output_hidden_states=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 1087, in forward hidden_states = self.embeddings(pixel_values, interpolate_pos_encoding=interpolate_pos_encoding) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(_args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 310, in forward patch_embeds = self.patch_embedding(pixelvalues) # shape = [, width, grid, grid] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\conv.py", line 460, in forward return self._conv_forward(input, self.weight, self.bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\conv.py", line 456, in _conv_forward return F.conv2d(input, weight, bias, self.stride, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Input type (torch.FloatTensor) and weight type (torch.HalfTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

[2024-10-19 15:20] Prompt executed in 2.21 seconds

Thanks

We appreciate the suggestion from @cchance27 . Please update to the latest version and add the following line to joy_caption_two_node.py, line 64:

     def encode_image(self, pixel_values):
        #print(f"{id(self)}之前 in JoyClipVisionModel: {next(self.model.parameters()).device}")  # 打印模型参数的设备
        load_models_gpu([self.patcher], force_full_load=True, force_patch_weights=True)
        #print(f"之后 in JoyClipVisionModel: {next(self.model.parameters()).device}")  # 打印模型参数的设备
        # Convert image data to the model's dtype
        pixel_values = pixel_values.to(dtype=self.type)
        vision_outputs = self.model(pixel_values=pixel_values, output_hidden_states=True)
        return vision_outputs

Specifically, add this line:

       # Convert image data to the model's dtype
        pixel_values = pixel_values.to(dtype=self.type)

Please restart ComfyUI and try again. I sincerely apologize for the inconvenience, but I am unable to reproduce this bug in my environment. I would be grateful if you could test this solution and let me know if it resolves the issue.

AALeslie2468 commented 1 month ago

unfortunately, new error massage,

I think I will give up, because I think it is maybe some unknown problems in my computer. I am very sorry for wasting your time, I will use the online version in "huggingface", Thank you very much for your help

I also give you the error message in the "comfyui.log" for you reference :

ComfyUI-Manager: installing dependencies done.

[2024-10-24 01:54:59.987] ComfyUI startup time: 2024-10-24 01:54:59.987980 [2024-10-24 01:55:00.010] Platform: Windows [2024-10-24 01:55:00.011] Python version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)] [2024-10-24 01:55:00.011] Python executable: D:\AI\ComfyUI_windows_portable\python_embeded\python.exe [2024-10-24 01:55:00.011] ComfyUI Path: D:\AI\ComfyUI_windows_portable\ComfyUI [2024-10-24 01:55:00.011] Log path: D:\AI\ComfyUI_windows_portable\comfyui.log [2024-10-24 01:55:00.020] Prestartup times for custom nodes: [2024-10-24 01:55:00.021] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy [2024-10-24 01:55:00.022] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use [2024-10-24 01:55:00.022] 1.8 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager [2024-10-24 01:55:00.022] Total VRAM 6144 MB, total RAM 32539 MB [2024-10-24 01:55:01.551] pytorch version: 2.2.2+cu121 [2024-10-24 01:55:01.552] Set vram state to: NORMAL_VRAM [2024-10-24 01:55:01.552] Device: cuda:0 NVIDIA GeForce GTX 1660 SUPER : cudaMallocAsync [2024-10-24 01:55:02.624] Using pytorch cross attention [2024-10-24 01:55:03.360] [Prompt Server] web root: D:\AI\ComfyUI_windows_portable\ComfyUI\web [2024-10-24 01:55:03.361] Adding extra search path checkpoints D:\AI\stable-diffusion-webui\models/Stable-diffusion [2024-10-24 01:55:03.361] Adding extra search path configs D:\AI\stable-diffusion-webui\models/Stable-diffusion [2024-10-24 01:55:03.361] Adding extra search path vae D:\AI\stable-diffusion-webui\models/VAE [2024-10-24 01:55:03.361] Adding extra search path loras D:\AI\stable-diffusion-webui\models/Lora [2024-10-24 01:55:03.361] Adding extra search path loras D:\AI\stable-diffusion-webui\models/LyCORIS [2024-10-24 01:55:03.361] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/ESRGAN [2024-10-24 01:55:03.361] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/RealESRGAN [2024-10-24 01:55:03.361] Adding extra search path upscale_models D:\AI\stable-diffusion-webui\models/SwinIR [2024-10-24 01:55:03.363] Adding extra search path embeddings D:\AI\stable-diffusion-webui\embeddings [2024-10-24 01:55:03.363] Adding extra search path hypernetworks D:\AI\stable-diffusion-webui\models/hypernetworks [2024-10-24 01:55:03.363] Adding extra search path controlnet D:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\models [2024-10-24 01:55:04.629] [Crystools INFO] Crystools version: 1.20.0 [2024-10-24 01:55:04.647] [Crystools INFO] CPU: 13th Gen Intel(R) Core(TM) i5-13500 - Arch: AMD64 - OS: Windows 10 [2024-10-24 01:55:04.657] [Crystools INFO] Pynvml (Nvidia) initialized. [2024-10-24 01:55:04.657] [Crystools INFO] GPU/s: [2024-10-24 01:55:04.664] [Crystools INFO] 0) NVIDIA GeForce GTX 1660 SUPER [2024-10-24 01:55:04.664] [Crystools INFO] NVIDIA Driver: 560.94 [2024-10-24 01:55:05.355] [ComfyUI-Easy-Use] server: v1.2.4 Loaded [2024-10-24 01:55:05.355] [ComfyUI-Easy-Use] web root: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use\web_version/v2 Loaded [2024-10-24 01:55:05.381] ### Loading: ComfyUI-Impact-Pack (V7.10.6) [2024-10-24 01:55:05.755] ### Loading: ComfyUI-Impact-Pack (Subpack: V0.7) [2024-10-24 01:55:05.783] [Impact Pack] Wildcards loading done. [2024-10-24 01:55:05.792] ### Loading: ComfyUI-Inspire-Pack (V1.6) [2024-10-24 01:55:05.836] ### Loading: ComfyUI-Manager (V2.51.8) [2024-10-24 01:55:06.004] ### ComfyUI Revision: 2781 [915fdb57] | Released on '2024-10-22' [2024-10-24 01:55:06.183] [SD Prompt Reader] Node version: 1.3.4 [2024-10-24 01:55:06.183] [SD Prompt Reader] Core version: 1.3.5 [2024-10-24 01:55:06.241] ------------------------------------------ [2024-10-24 01:55:06.241] Comfyroll Studio v1.76 :  175 Nodes Loaded [2024-10-24 01:55:06.241] ------------------------------------------ [2024-10-24 01:55:06.241] For changes, please see patch notes at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/blob/main/Patch_Notes.md [2024-10-24 01:55:06.241] For help, please see the wiki at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/wiki [2024-10-24 01:55:06.242] ------------------------------------------ [2024-10-24 01:55:06.254] [comfyui_controlnet_aux] | INFO -> Using ckpts path: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts [2024-10-24 01:55:06.254] [comfyui_controlnet_aux] | INFO -> Using symlinks: False [2024-10-24 01:55:06.254] [comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider'] [2024-10-24 01:55:06.273] D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\node_wrappers\dwpose.py:26: UserWarning: DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly warnings.warn("DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly") [2024-10-24 01:55:06.341] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json [2024-10-24 01:55:06.351] Efficiency Nodes: Attempting to add Control Net options to the 'HiRes-Fix Script' Node (comfyui_controlnet_aux add-on)...Success! [2024-10-24 01:55:06.395] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json [2024-10-24 01:55:06.434] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json [2024-10-24 01:55:06.480] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json [2024-10-24 01:55:06.512] [2024-10-24 01:55:06.514] [rgthree-comfy] Loaded 42 exciting nodes. 🎉 [2024-10-24 01:55:06.514] [2024-10-24 01:55:06.540] [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [2024-10-24 01:55:07.694] WAS Node Suite: OpenCV Python FFMPEG support is enabled [2024-10-24 01:55:07.694] WAS Node Suite Warning: ffmpeg_bin_path is not set in D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\was_suite_config.json config file. Will attempt to use system ffmpeg binaries if available. [2024-10-24 01:55:08.655] WAS Node Suite: Finished. Loaded 218 nodes successfully. [2024-10-24 01:55:08.656] "Art enables us to find ourselves and lose ourselves at the same time." - Thomas Merton [2024-10-24 01:55:08.656] [2024-10-24 01:55:08.667] Import times for custom nodes: [2024-10-24 01:55:08.667] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\websocket_image_save.py [2024-10-24 01:55:08.667] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Noise [2024-10-24 01:55:08.667] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale [2024-10-24 01:55:08.667] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-GGUF [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Derfuu_ComfyUI_ModdedNodes [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-prompt-reader-node [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_bitsandbytes_NF4 [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack [2024-10-24 01:55:08.668] 0.0 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux [2024-10-24 01:55:08.668] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Crystools [2024-10-24 01:55:08.668] 0.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\facerestore_cf [2024-10-24 01:55:08.669] 0.3 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager [2024-10-24 01:55:08.669] 0.4 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack [2024-10-24 01:55:08.669] 0.7 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Easy-Use [2024-10-24 01:55:08.669] 2.1 seconds: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui [2024-10-24 01:55:08.669] [2024-10-24 01:55:08.680] Starting server

[2024-10-24 01:55:08.681] To see the GUI go to: http://127.0.0.1:8188 [2024-10-24 01:55:09.992] FETCH DATA from: D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE] [2024-10-24 01:55:10.742] [Inspire Pack] IPAdapterPlus is not installed. [2024-10-24 01:55:49.091] got prompt [2024-10-24 01:55:49.139] D:\AI\ComfyUI_windows_portable\ComfyUI\models\clip\siglip-so400m-patch14-384 [2024-10-24 01:55:51.146] Loading VLM's custom vision model [2024-10-24 01:55:54.294] Requested to load SiglipVisionTransformer [2024-10-24 01:55:54.294] Loading 1 new model [2024-10-24 01:55:54.300] loaded completely 0.0 809.1729736328125 True [2024-10-24 01:55:55.032] !!! Exception during processing !!! "LayerNormKernelImpl" not implemented for 'Half' [2024-10-24 01:55:55.037] Traceback (most recent call last): File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "D:\AI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs results.append(getattr(obj, func)(inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two\joy_caption_two_node.py", line 343, in generate vision_outputs = joy_two_pipeline.clip_model.encode_image(pixel_values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two\joy_caption_two_node.py", line 70, in encode_image vision_outputs = self.model(pixel_values=pixel_values, output_hidden_states=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 1089, in forward encoder_outputs = self.encoder( ^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 900, in forward layer_outputs = encoder_layer( ^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\siglip\modeling_siglip.py", line 641, in forward hidden_states = self.layer_norm1(hidden_states) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\normalization.py", line 201, in forward return F.layer_norm( ^^^^^^^^^^^^^ File "D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\functional.py", line 2546, in layer_norm return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

[2024-10-24 01:55:55.039] Prompt executed in 5.94 seconds

EvilBT commented 4 weeks ago

Hi! After analyzing the error, it seems the issue is stemming from the limited FP16 support on your GTX 1660 SUPER graphics card. To address this, please try making a small manual code change:

Open the file joy_caption_two_node.py.

Locate the following lines of code (around line 37):

class JoyClipVisionModel: def __init__(self, load_device, offload_device): self.load_device = load_device self.offload_device = offload_device self.type = text_encoder_dtype() # <--- Change this line

Change the line self.type = text_encoder_dtype() to:

self.type = self.type = comfy.model_management.unet_dtype(self.load_device) This modification will automatically select the appropriate computation precision based on your graphics card's capabilities, preventing the "RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'" error.

After making this change, please save the file and restart ComfyUI. Then, try running your workflow again. Let me know if the issue persists after making this change, and I'll do my best to help you further!