sipie800 / ComfyUI-PuLID-Flux-Enhanced

Apache License 2.0
39 stars 5 forks source link

No longer works since latest Comfy Update #25

Open Pantonia4 opened 4 days ago

Pantonia4 commented 4 days ago

Since todays update, PulID is no longer working on my end. Doing a rollback to the previous version makes it work again.

The Error:

!!! Exception during processing !!! forward_orig() takes from 7 to 9 positional arguments but 10 were given
Traceback (most recent call last):
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy_extras\nodes_custom_sampler.py", line 633, in sample
    samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise.seed)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\samplers.py", line 740, in sample
    output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\samplers.py", line 719, in inner_sample
    samples = sampler.sample(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\samplers.py", line 624, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\k_diffusion\sampling.py", line 155, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\samplers.py", line 299, in __call__
    out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\samplers.py", line 706, in __call__
    return self.predict_noise(*args, **kwargs)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\samplers.py", line 709, in predict_noise
    return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\samplers.py", line 279, in sampling_function
    out = calc_cond_batch(model, conds, x, timestep, model_options)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\samplers.py", line 228, in calc_cond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\model_base.py", line 144, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\AI\StabilityMatrix\Packages\ComfyUI\comfy\ldm\flux\model.py", line 181, in forward
    out = self.forward_orig(img, img_ids, context, txt_ids, timestep, y, guidance, control, transformer_options)
TypeError: forward_orig() takes from 7 to 9 positional arguments but 10 were given
adydeejay commented 4 days ago

It seems that the problem comes from this line. I modified it back to this and now image generation works with PuLID.

Please note that this is a temporary workaround for simple workflows with PuLID and is not a permanent fix; it may also impact the functionality of other nodes.

fengwy commented 4 days ago

same problem

mbac commented 3 days ago

Same here

Malie73 commented 3 days ago

same problem, workaround does its job but please fix it.

mbac commented 3 days ago

Maybe this is not the best place for this, but: can people please post here about any adverse effects they incur as a result of changing that line of code?

stranger-games commented 3 days ago

I kept reverting comfyui to previous commits and reached the commit 8b90e50979b0d33e1f8d10d5c938361f59f95474 but still it's not working. @Pantonia4 Can you tell us which commit exactly broke this custom node?


In my case I just did the following which was referenced in another issue and all works well with the latest commit. So the error is not the same.

python.exe -s -m pip install --use-pep517 facexlib

OK, that was another error indeed but I still got the error mentioned here. I tried PR #27 which fixes this issue but in my case gives another error probably because of control net.


Actually I confirmed that it's not the control net, the PR seems to need an update.

BerglinJ commented 11 hours ago

Use together with Hyper-FLUX.1-dev-8step-lora is not working either. Only get noise image

adydeejay commented 39 minutes ago

It seems that the problem comes from this line. I modified it back to this and now image generation works with PuLID.

Please note that this is a temporary workaround for simple workflows with PuLID and is not a permanent fix; it may also impact the functionality of other nodes.

For me, the PuLID node stopped working immediately after I updated Comfyui on 2024-11-12. So, after @Pantonia4 posted the error "TypeError: forward_orig() takes from 7 to 9 positional arguments but 10 were given" in the Comfyui repository, considering that the PuLID nodes had not undergone changes, I started looking for what it was updated in Comfyui.

First I checked the branches. https://github.com/comfyanonymous/ComfyUI/branches https://github.com/comfyanonymous/ComfyUI/tree/patch_hooks_improved_memory https://github.com/comfyanonymous/ComfyUI/pull/5583 https://github.com/comfyanonymous/ComfyUI/tree/improved_memory https://github.com/comfyanonymous/ComfyUI/compare/improved_memory...master click on Files changed search for "flux" and "comfy/ldm/flux/model.py"

https://github.com/comfyanonymous/ComfyUI/compare/improved_memory...master#diff-453e6e6c1a6fa355aa5f5b6aabb5f9153c526c115d08ad757c0243d9f8ca31ee

and I took each line in turn, so I discovered that removing transformer_options from ..ComfyUI_windows_portable\ComfyUI\comfy\ldm\flux\model.py solves the problem (at least for me).

out = self.forward_orig(img, img_ids, context, txt_ids, timestep, y, guidance, control, transformer_options)

https://github.com/comfyanonymous/ComfyUI/commit/96b2080971b1e6606bdbaf2aa4f24b7f aca3999a#diff-453e6e6c1a6fa355aa5f5b6aabb5f9153c526c115d08ad757c0243d9f8ca31eeL159

Maybe someone much more skilled will find a better solution - I'm not a coder.

Since 2024-11-12 I haven't updated Comfyui or any other nodes, and it seems that now there is a fix for forward_orig: https://github.com/sipie800/ComfyUI-PuLID-Flux-Enhanced/pull/27#issue-2664002771

Using FLUX-Dev I get ~1 min (2.10s/it) to generate an 1080p image with PuLID and ~3 min to generate a video sequence 1080p 60fps (10.78s/it, dpmpp/karras/cfg2.5) using SVD/Video Helper Suite. Sometimes the time increases to 5-15 min per img/vid so I have to cleanup the VRAM or unload modules or to restart Comfyui to get the same working speed. But I personally find it extremely acceptable.

OK. Now I am trying to finish a personal project, but I hope that during the next week I will upgrade Comfyui and the nodes, I will test more and come back with more details. Also, I suspect a problem with the Nvidia driver and CUDA package on this torch version ... 2.4.1+cu121 was much faster. So... there will be more work.

My own specs + log, packages list and snapshot.

Platform: Windows 10 Pro 64-bit v.22H2 Python version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)] Total VRAM 24564 MB, total RAM 130980 MB pytorch version: 2.5.1+cu124 xformers version: 0.0.28.post3 Set vram state to: NORMAL_VRAM Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync GPU/s: NVIDIA GeForce RTX 4090 NVIDIA Driver: 566.03 ComfyUI-Manager (V2.51.9) ComfyUI Revision: 2827 [8ebf2d88] | Released on '2024-11-12'

comfyui.log packages list.txt 2024-11-17_23-59-18_snapshot.json

I hope this information helps you. Thanks everyone for all the great work!