Nuked88 / ComfyUI-N-Nodes

A suite of custom nodes for ConfyUI that includes GPT text-prompt generation, LoadVideo, SaveVideo, LoadFramesFromFolder and FrameInterpolator
MIT License
205 stars 22 forks source link

GPTLoaderSimple and GPTSampler | Node conflict #27

Closed dummerfranz closed 11 months ago

dummerfranz commented 11 months ago

Conflict with other nodes. GPTLoaderSimple and GPTSampler is conflicting with other nodes for me. im not able to generate images with Ksampler if GPT Nodes are in the same workflow.

Error occurred when executing Efficient Loader: Cannot copy out of meta tensor; no data!

Error occurred when executing CLIPTextEncode: Cannot copy out of meta tensor; no data!

Nuked88 commented 11 months ago

Without a full log (all the text in the console from the time ComfyUI was started until the time of the reported bug) i will be not able to help you in any way man.

dummerfranz commented 11 months ago

Screenshot 2023-12-22 185355

`Error occurred when executing Efficient Loader:

Cannot copy out of meta tensor; no data!

File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 172, in efficientloader encode_prompts(positive, negative, token_normalization, weight_interpretation, clip, clip_skip, File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 73, in encode_prompts positive_encoded = bnk_adv_encode.AdvancedCLIPTextEncode().encode(clip, positive_prompt, token_normalization, weight_interpretation)[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 312, in encode embeddings_final, pooled = advanced_encode(clip, text, token_normalization, weight_interpretation, w_max=1.0, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 246, in advanced_encode embsl, = advanced_encode_from_tokens(tokenized['l'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 183, in advanced_encode_from_tokens weighted_emb, pooled_base = encode_func(weighted_tokens) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 249, in lambda x: encode_token_weights(clip, x, encode_token_weights_l), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 229, in encode_token_weights return encode_func(model.cond_stage_model, token_weight_pairs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 221, in encode_token_weights_l lout, = model.clip_l.encode_token_weights(token_weight_pairs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 40, in encode_token_weights out, pooled = self.encode(to_encode) ^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 191, in encode return self(tokens) ^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 173, in forward outputs = self.transformer(tokens, attention_mask, intermediate_output=self.layer_idx, final_layer_norm_intermediate=self.layer_norm_hidden_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\hooks.py", line 159, in new_forward args, kwargs = module._hf_hook.pre_forward(module, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\hooks.py", line 290, in pre_forward return send_to_device(args, self.execution_device), send_to_device( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\utils\operations.py", line 151, in send_to_device return honor_type( ^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\utils\operations.py", line 83, in honor_type return type(obj)(generator) ^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\utils\operations.py", line 152, in tensor, (send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys) for t in tensor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\utils\operations.py", line 167, in send_to_device return tensor.to(device, non_blocking=non_blocking) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^`

Nuked88 commented 11 months ago

From the start! 😋 Also, and i'm asking, efficent loader doesn't have an sdxl dedicated node?

dummerfranz commented 11 months ago

The error is happening if multiple GPTLoaderSimple or GPTSampler are present in the workflow. I believe the order of which node is running first is messed up.

Screenshot 2023-12-22 192906

CONSOLE

D:\ComfyUI_windows_portable>.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build ComfyUI startup time: 2023-12-22 19:27:41.519784 Platform: Windows Python version: 3.11.6 (tags/v3.11.6:8b6ee5b, Oct 2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)] Python executable: D:\ComfyUI_windows_portable\python_embeded\python.exe ** Log path: D:\ComfyUI_windows_portable\comfyui.log

Prestartup times for custom nodes: 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager

Total VRAM 8192 MB, total RAM 32653 MB Set vram state to: NORMAL_VRAM Device: cuda:0 NVIDIA GeForce RTX 3070 : cudaMallocAsync VAE dtype: torch.bfloat16 Using pytorch cross attention Adding extra search path checkpoints models/ [Allor]: 0 nodes were overridden. [Allor]: 12 modules were enabled. [Allor]: 98 nodes were loaded.

Loading: ComfyUI-Impact-Pack (V4.48.12)

Loading: ComfyUI-Impact-Pack (Subpack: V0.3.2)

[Impact Pack] Wildcards loading done.

Loading: ComfyUI-Manager (V1.15.1)

ComfyUI Revision: 1848 [d35267e8] | Released on '2023-12-21'

FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json packaging is already installed. FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json cpuinfo is already installed. git is already installed. moviepy is already installed. cv2 is already installed. [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json [ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json skbuild is already installed. typing is already installed. diskcache is already installed. llama_cpp is already installed.

Total VRAM 8192 MB, total RAM 32653 MB Set vram state to: NORMAL_VRAM Device: cuda:0 NVIDIA GeForce RTX 3070 : cudaMallocAsync VAE dtype: torch.bfloat16 Torch version: 2.1.1+cu121

Comfyroll Custom Nodes: 148 Nodes Loaded

[comfyui_controlnet_aux] | INFO -> Using ckpts path: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts [comfyui_controlnet_aux] | INFO -> Using symlinks: False [comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider'] DWPose: Onnxruntime with acceleration providers detected FizzleDorf Custom Nodes: Loaded [comfy_mtb] | INFO -> loaded 58 nodes successfuly [comfy_mtb] | INFO -> Some nodes (4) could not be loaded. This can be ignored, but go to http://127.0.0.1:8188/mtb if you want more information. Efficiency Nodes: Attempting to add Control Net options to the 'HiRes-Fix Script' Node (comfyui_controlnet_aux add-on)...Success! Efficiency Nodes: Attempting to add 'AnimatedDiff Script' Node (ComfyUI-AnimateDiff-Evolved add-on)...Success!

[rgthree] Loaded 20 exciting nodes. [rgthree] Optimizing ComfyUI recursive execution. If queueing and/or re-queueing seems broken, change "patch_recursive_execution" to false in rgthree_config.json

WAS Node Suite: OpenCV Python FFMPEG support is enabled WAS Node Suite Warning: ffmpeg_bin_path is not set in D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\was_suite_config.json config file. Will attempt to use system ffmpeg binaries if available. WAS Node Suite: Finished. Loaded 198 nodes successfully.

    "Don't wait. The time will never be just right." - Napoleon Hill

Import times for custom nodes: 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\rembg-comfyui-node 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\sdxl_prompt_styler 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_GradientDeepShrink 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\sdxl-recommended-res-calc 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\cg-use-everywhere 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Logic 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\masquerade-nodes-comfyui 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-post-processing-nodes 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\stability-ComfyUI-nodes 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\steerable-motion 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfy-image-saver 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Frame-Interpolation 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\Derfuu_ComfyUI_ModdedNodes 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_essentials 0.0 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes 0.2 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_FizzNodes 0.3 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux 0.3 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-N-Nodes 0.3 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfy_mtb 0.4 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-reactor-node 0.4 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_segment_anything 0.4 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager 0.4 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack 0.7 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Allor 1.4 seconds: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui

Starting server

To see the GUI go to: http://127.0.0.1:8188 FETCH DATA from: D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json got prompt 6 {'id': 'cmpl-bde4cf04-7ef1-43f0-84ef-fff7e262e330', 'object': 'text_completion', 'created': 1703269681, 'model': 'D:\ComfyUI_windows_portable\ComfyUI\models\GPTcheckpoints\mistral-7b-instruct-v0.2.Q4_K_M.gguf', 'choices': [{'text': '🌷 "Feeling grateful for the little things today. A cup of tea, a cozy blanket, and this beautiful bloom that just appeared on my windowsill. Sometimes the simplest pleasures bring us the most joy. #selfcare #gratitude"', 'index': 0, 'logprobs': None, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 17, 'completion_tokens': 57, 'total_tokens': 74}} Requested to load SDXLClipModel Loading 1 new model Using pytorch attention in VAE Working with z of shape (1, 4, 32, 32) = 4096 dimensions. Using pytorch attention in VAE Leftover VAE keys ['model_ema.decay', 'model_ema.num_updates']

Efficient Loader Models Cache: Ckpt: [1] dreamshaperXL_turboDpmppSDE Vae: [1] sdxl_vae Requested to load SDXL Loading 1 new model loading in lowvram mode 1766.4303407669067 100%|████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:20<00:00, 3.37s/it] Requested to load AutoencoderKL Loading 1 new model You shouldn't move a model when it is dispatched on multiple devices. 6 {'id': 'cmpl-aedc782f-69b2-4785-9b82-30ef22f590d3', 'object': 'text_completion', 'created': 1703269712, 'model': 'D:\ComfyUI_windows_portable\ComfyUI\models\GPTcheckpoints\mistral-7b-instruct-v0.2.Q4_K_M.gguf', 'choices': [{'text': '🌱💚 "Nurturing new beginnings. Every day is an opportunity to grow, learn and bloom. #selfcare #mindfulness #newbeginnings" #plantlove #gardenlife #growthmindset', 'index': 0, 'logprobs': None, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 17, 'completion_tokens': 52, 'total_tokens': 69}} Input: write me a short instagram post Generated Text: 🌱💚 "Nurturing new beginnings. Every day is an opportunity to grow, learn and bloom. #selfcare #mindfulness #newbeginnings" #plantlove #gardenlife #growthmindset Requested to load SDXLClipModel Loading 1 new model loading in lowvram mode 256.0 ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 172, in efficientloader encode_prompts(positive, negative, token_normalization, weight_interpretation, clip, clip_skip, File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 73, in encode_prompts positive_encoded = bnk_adv_encode.AdvancedCLIPTextEncode().encode(clip, positive_prompt, token_normalization, weight_interpretation)[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 312, in encode embeddings_final, pooled = advanced_encode(clip, text, token_normalization, weight_interpretation, w_max=1.0, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 246, in advanced_encode embsl, = advanced_encode_from_tokens(tokenized['l'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 183, in advanced_encode_from_tokens weighted_emb, pooled_base = encode_func(weighted_tokens) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 249, in lambda x: encode_token_weights(clip, x, encode_token_weights_l), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 229, in encode_token_weights return encode_func(model.cond_stage_model, token_weight_pairs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 221, in encode_token_weights_l lout, = model.clip_l.encode_token_weights(token_weight_pairs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 40, in encode_token_weights out, pooled = self.encode(to_encode) ^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 191, in encode return self(tokens) ^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 173, in forward outputs = self.transformer(tokens, attention_mask, intermediate_output=self.layer_idx, final_layer_norm_intermediate=self.layer_norm_hidden_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\hooks.py", line 159, in new_forward args, kwargs = module._hf_hook.pre_forward(module, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\hooks.py", line 290, in pre_forward return send_to_device(args, self.execution_device), send_to_device( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\utils\operations.py", line 151, in send_to_device return honor_type( ^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\utils\operations.py", line 83, in honor_type return type(obj)(generator) ^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\utils\operations.py", line 152, in tensor, (send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys) for t in tensor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\accelerate\utils\operations.py", line 167, in send_to_device return tensor.to(device, non_blocking=non_blocking) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ NotImplementedError: Cannot copy out of meta tensor; no data!

Prompt executed in 47.10 seconds

WORKFLOW

TEST (5).json

Nuked88 commented 11 months ago

Thanks man i will check it out! I think that "You shouldn't move a model when it is dispatched on multiple devices." can be a clue on the issues but i need verify that.

Nuked88 commented 11 months ago

So, i tryed to load your workflow but it was looking empty to me so i've create a new one almost identical. As expected i didn't have any "You shouldn't move a model when it is dispatched on multiple devices." and it was running totally fine. I noticed that you have: "loading in lowvram mode " idk if is an automatic thing of efficent loader or you are running comfyui with the lowvram flag. If is an automatic thing i think that this is the issue: GPT node is loading the model in your VRAM, Efficent Loader is tring to load the model also in your VRAM but since is almost full it will try to run the system in lowvram mode causing this issue. So i think (again, if you are NOT running comfyui with the --lowvram flag) that is an efficent loader bug or something like that. If you want you can reduce the gpu_layers in the GPT Loader Simple so it will consume alot less VRAM, with 0 it will not consume VRAM at all but it will be alot slower. Here my test workflow: workflow

Please let me know

dummerfranz commented 11 months ago

"GPT node is loading the model in your VRAM, Efficent Loader is tring to load the model also in your VRAM but since is almost full it will try to run the system in lowvram mode causing this issue." -> you are right. i had multiple GPT loader in my workflow with the same model. that pushed my vram to its limit. --lowvram flag was not used. "loading in lowvram mode" - is triggered automatically on fresh confyui install. now its running fine

Nuked88 commented 11 months ago

Perfect! Thanks for the info!