cubiq / ComfyUI_IPAdapter_plus

GNU General Public License v3.0
3.17k stars 238 forks source link

Style transfer broke after last update #460

Closed shrewen closed 2 months ago

shrewen commented 2 months ago

This last update broke the style-transfer for me, now all I'm getting is this error, I'm running Windows 11 on an NVIDIX GTX 1660S and a 5900x AMD RYZEN CPU, if that makes a difference.

`Error occurred when executing KSampler:

unsupported operand type(s) for *: 'Tensor' and 'dict'

File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\nodes.py", line 1344, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\nodes.py", line 1314, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample return original_sample(args, kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\sample.py", line 37, in sample samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 755, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 657, in sample return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 644, in sample output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 623, in inner_sample samples = sampler.sample(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 534, in sample samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, self.extra_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\k_diffusion\sampling.py", line 539, in sample_dpmpp_sde denoised = model(x, sigmas[i] * s_in, extra_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 272, in call out = self.inner_model(x, sigma, model_options=model_options, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 610, in call return self.predict_noise(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 613, in predict_noise return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 258, in sampling_function out = calc_cond_batch(model, conds, x, timestep, model_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\samplers.py", line 218, in calc_cond_batch output = model.apply_model(inputx, timestep, c).chunk(batch_chunks) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\model_base.py", line 97, in apply_model model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, *extra_conds).float() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 850, in forward h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 44, in forward_timestep_embed x = layer(x, context, transformer_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\ldm\modules\attention.py", line 633, in forward x = block(x, context=context[i], transformer_options=transformer_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\ldm\modules\attention.py", line 460, in forward return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\ldm\modules\diffusionmodules\util.py", line 191, in checkpoint return func(inputs) ^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\comfy\ldm\modules\attention.py", line 557, in _forward n = attn2_replace_patch[block_attn2](n, context_attn2, value_attn2, extra_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER-A\Documents\Portables\ComfyUI\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\CrossAttentionPatch.py", line 175, in call out_ip = out_ip * weight # I'm doing this to get the same results as before ~^~`

cubiq commented 2 months ago

please try the style transfer example in this repo and report back

shrewen commented 2 months ago

The Style and Composition node works, only the style-transfer weight in the regular ipadapter node throw the error I sent you.

Kind regards, Ramy


From: Matteo Spinelli @.> Sent: 12 April 2024 17:49 To: cubiq/ComfyUI_IPAdapter_plus @.> Cc: shrewen @.>; Author @.> Subject: Re: [cubiq/ComfyUI_IPAdapter_plus] Style transfer broke after last update (Issue #460)

please try the style transfer example in this repo and report back

— Reply to this email directly, view it on GitHubhttps://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/460#issuecomment-2052016347, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIN3ABDKWKOUZKPZJ4ENZATY477BVAVCNFSM6AAAAABGEJ4N6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGAYTMMZUG4. You are receiving this because you authored the thread.Message ID: @.***>

cubiq commented 2 months ago

are you using instantid? try to update both

shrewen commented 2 months ago

I was using it with another ipadapter node connected to a faceid unified node, but even with one ipadapter node the style-transfer throws the error, and everything is indeed up to date


From: Matteo Spinelli @.> Sent: 12 April 2024 18:25 To: cubiq/ComfyUI_IPAdapter_plus @.> Cc: shrewen @.>; Author @.> Subject: Re: [cubiq/ComfyUI_IPAdapter_plus] Style transfer broke after last update (Issue #460)

are you using instantid? try to update both

— Reply to this email directly, view it on GitHubhttps://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/460#issuecomment-2052080631, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIN3ABE7KAQY3LCBGTWBWJDY5ADITAVCNFSM6AAAAABGEJ4N6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGA4DANRTGE. You are receiving this because you authored the thread.Message ID: @.***>

cubiq commented 2 months ago

can you please try to upgrade both again, I just tried and everything seems to be working with the latest version

shrewen commented 2 months ago

I tried updating again but no update was found, and still the same error


From: Matteo Spinelli @.> Sent: 12 April 2024 18:29 To: cubiq/ComfyUI_IPAdapter_plus @.> Cc: shrewen @.>; Author @.> Subject: Re: [cubiq/ComfyUI_IPAdapter_plus] Style transfer broke after last update (Issue #460)

can you please try to upgrade both again, I just tried and everything seems to be working with the latest version

— Reply to this email directly, view it on GitHubhttps://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/460#issuecomment-2052086640, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIN3ABBN3TRJDRDILHXSVB3Y5ADXHAVCNFSM6AAAAABGEJ4N6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGA4DMNRUGA. You are receiving this because you authored the thread.Message ID: @.***>

cubiq commented 2 months ago

can you try with a clean workflow from the examples? like the "advanced" and change to style_transfer weight type

shrewen commented 2 months ago

The advanced node worked, so the issue is within the unified node


From: Matteo Spinelli @.> Sent: 12 April 2024 18:37 To: cubiq/ComfyUI_IPAdapter_plus @.> Cc: shrewen @.>; Author @.> Subject: Re: [cubiq/ComfyUI_IPAdapter_plus] Style transfer broke after last update (Issue #460)

can you try with a clean workflow from the examples? like the "advanced" and change to style_transfer weight type

— Reply to this email directly, view it on GitHubhttps://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/460#issuecomment-2052097655, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIN3ABCRJB6FWOBLRTCZXVTY5AEUDAVCNFSM6AAAAABGEJ4N6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGA4TONRVGU. You are receiving this because you authored the thread.Message ID: @.***>

cubiq commented 2 months ago

the only way for me to escalate is to have a "minimal" workflow that triggers the error. basically a workflow without any additional extensions except for ipadapter (and instantid if needed)

shrewen commented 2 months ago

Here is a minimal workflow that reproduces the error


From: Matteo Spinelli @.> Sent: 12 April 2024 18:46 To: cubiq/ComfyUI_IPAdapter_plus @.> Cc: shrewen @.>; Author @.> Subject: Re: [cubiq/ComfyUI_IPAdapter_plus] Style transfer broke after last update (Issue #460)

the only way for me to escalate is to have a "minimal" workflow that triggers the error. basically a workflow without any additional extensions except for ipadapter (and instantid if needed)

— Reply to this email directly, view it on GitHubhttps://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/460#issuecomment-2052111355, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIN3ABFSPE22FMJYRXW4KX3Y5AFW5AVCNFSM6AAAAABGEJ4N6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGEYTCMZVGU. You are receiving this because you authored the thread.Message ID: @.***>

cubiq commented 2 months ago

nothing is attached

shrewen commented 2 months ago

It's attached but I'll try again

On Fri, Apr 12, 2024 at 6:54 PM Matteo Spinelli @.***> wrote:

nothing is attached

— Reply to this email directly, view it on GitHub https://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/460#issuecomment-2052122950, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIN3ABCBB4FOIJ5KS7IKGBLY5AGSVAVCNFSM6AAAAABGEJ4N6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGEZDEOJVGA . You are receiving this because you authored the thread.Message ID: @.***>

cubiq commented 2 months ago

you can't send attachments via email to github

shrewen commented 2 months ago

Oh ok, here is a drive link instead, sorry for the inconvenience

https://drive.google.com/file/d/1JBeZJSN0L6G-8HmfYopj5Dzr_JwjSPra/view?usp=drive_link


From: Matteo Spinelli @.> Sent: 12 April 2024 18:58 To: cubiq/ComfyUI_IPAdapter_plus @.> Cc: shrewen @.>; Author @.> Subject: Re: [cubiq/ComfyUI_IPAdapter_plus] Style transfer broke after last update (Issue #460)

you can't send attachments via email to github

— Reply to this email directly, view it on GitHubhttps://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/460#issuecomment-2052131047, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIN3ABF66ADMKOSTTXP6XZ3Y5AHEBAVCNFSM6AAAAABGEJ4N6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGEZTCMBUG4. You are receiving this because you authored the thread.Message ID: @.***>

cubiq commented 2 months ago

this is fixed

shrewen commented 2 months ago

Cheers matteo! all is working as it should


From: Matteo Spinelli @.> Sent: 12 April 2024 19:16 To: cubiq/ComfyUI_IPAdapter_plus @.> Cc: shrewen @.>; Author @.> Subject: Re: [cubiq/ComfyUI_IPAdapter_plus] Style transfer broke after last update (Issue #460)

this is fixed

— Reply to this email directly, view it on GitHubhttps://github.com/cubiq/ComfyUI_IPAdapter_plus/issues/460#issuecomment-2052164604, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIN3ABDSJ6JINWU6XIJRRE3Y5AJHTAVCNFSM6AAAAABGEJ4N6GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGE3DINRQGQ. You are receiving this because you authored the thread.Message ID: @.***>

hben35096 commented 2 months ago

Thanks