shiimizu / ComfyUI_smZNodes

Custom nodes for ComfyUI such as CLIP Text Encode++
GNU General Public License v3.0
190 stars 14 forks source link

AND keyword does not work. Composable Diffusion via multi-conditioning does not function. #35

Open Quantumdrone opened 8 months ago

Quantumdrone commented 8 months ago

When prompting with a red abstract artwork The result is this: 045_PRE-Upscale_BrightProtoNuke1 2_dpmpp_3m_sde_karras_9_6 0 When prompting with a red abstract artwork AND boy standing on an ancient machine The result is this: 049_PRE-Upscale_BrightProtoNuke1 2_dpmpp_3m_sde_karras_9_6 0

BREAK works just fine. This is all with using the correct node "CLIP Text Encode++" and the parser set to "A1111".

I'd love to have this in my toolset, as it's the feature I miss the most out of Auto's build. Thanks for your work thus far in the creation of these nodes!

shiimizu commented 8 months ago

Thanks for bringing this to my attention. I found that it works if you enable CFGDenoiser in the Settings node, but that comes with its own limitations since model patches, controlnets, etc. currently don't work with that enabled. It'll take some time before that feature is fully fleshed out.

shiimizu commented 8 months ago

Alright, I finally fixed it in https://github.com/shiimizu/ComfyUI_smZNodes/commit/312aed3226305b63c4c758deef88d88832797589

Quantumdrone commented 8 months ago

Thanks for your hard work! It works partly, but throws me an error now when trying to generate a higher res image (via Ksampler) with an upscale model:


got prompt
Requested to load SDXL
Loading 1 new model
  0%|                                                                                                                                                                                                                | 0/12 [00:00<?, ?it/s]
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
  File "D:\0\ComfyUI\ComfyUI\execution.py", line 153, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\execution.py", line 83, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\execution.py", line 76, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\nodes.py", line 1299, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\nodes.py", line 1269, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 22, in informative_sample
    raise e
  File "D:\0\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample
    return original_sample(*args, **kwargs)  # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations.
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\comfy\sample.py", line 101, in sample
    samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\custom_nodes\ComfyUI_smZNodes\__init__.py", line 129, in KSampler_sample
    return _KSampler_sample(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\comfy\samplers.py", line 716, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\custom_nodes\ComfyUI_smZNodes\__init__.py", line 138, in sample
    return _sample(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\comfy\samplers.py", line 622, in sample
    samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\comfy\samplers.py", line 561, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\comfy\k_diffusion\sampling.py", line 580, in sample_dpmpp_2m
    denoised = model(x, sigmas[i] * s_in, **extra_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\comfy\samplers.py", line 285, in forward
    out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\comfy\samplers.py", line 275, in forward
    return self.apply_model(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\custom_nodes\ComfyUI_smZNodes\smZNodes.py", line 1015, in apply_model
    out = sampling_function(self.inner_model, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\custom_nodes\ComfyUI_smZNodes\smZNodes.py", line 1030, in sampling_function
    outt = comfy.samplers.calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\comfy\samplers.py", line 224, in calc_cond_uncond_batch
    output = model_options['model_function_wrapper'](model.apply_model, {"input": input_x, "timestep": timestep_, "c": c, "cond_or_uncond": cond_or_uncond}).chunk(batch_chunks)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\0\ComfyUI\ComfyUI\custom_nodes\ComfyUI_smZNodes\modules\sd_samplers_cfg_denoiser.py", line 178, in forward_
    uncond = torch.cat([torch.stack(chunk) for chunk in zip(*u_chunks)])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: torch.cat(): expected a non-empty list of Tensors
Quantumdrone commented 8 months ago

Somehow I can't see your response, though I did get a email notification with your asking for a workflow. Here is an image that includes it. Please let me know if you need anything else or the workflow in a different manner! Thanks ComfyUI_00001_

shiimizu commented 8 months ago

Thanks for the workflow. https://github.com/shiimizu/ComfyUI_smZNodes/commit/75c694182cc82d921900ba068c0dedc7253013c6 should havve it fixed

Quantumdrone commented 8 months ago

Thank you! AND does indeed work now. I'm terribly sorry to keep coming with new problems but there are a pair of new ones now. First, I'm getting an error per each step when having AND in the upscale portion. The generation still completes but I'm not sure if the error is effecting the end result: ERROR: index 1 is out of bounds for dimension 0 with size 1 Here is the workflow: ComfyUI_00004_

Secondly, BREAK doesn't work at all now. It gives an error message that ends with: RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x4096 and 2816x1280) I'd include the workflow but it doesn't even finish. If you just replace "AND" with "BREAK" in the workflow above, then you'll duplicate the exact workflow that causes the error. Thanks for your continued work, and happy holidays to you!

shiimizu commented 8 months ago

Happy holidays to you too! No worries, I appreciate your help. Thanks for the workflow again, it helped me pinpoint the issue. All the bugs mentioned should be fixed in the recent commits: https://github.com/shiimizu/ComfyUI_smZNodes/commit/5d3e7ede982b9020e07770f7018d03f79bbe3ba2 https://github.com/shiimizu/ComfyUI_smZNodes/commit/4a5d1d7d4517db8561403c5b7304c015f7507150 https://github.com/shiimizu/ComfyUI_smZNodes/commit/c9ac3a9d58755ebb53ca3c2e39472ea89d236e84. Happy prompting!