search

longgui0318 / comfyui-magic-clothing

ShineChen1024/MagicClothing comfyui support .
24 stars 4 forks source link

MagicClothing with IPAdapter #7

Open fAIseh00d opened 4 weeks ago

fAIseh00d commented 4 weeks ago

Hello!

I encountered attention patching errors when trying to use MagicClothing with IPAdapter, the logs are under spoilers below, here's the workflow.

Theoretically it should be possible to use MagicClothing in conjunction with IPAdapter, it's mentioned in the original paper, so it's must be some implementation issue or conflict with ComfyUI_IPAdapter_plus

Error log 1 ``` Error occurred when executing KSampler: shape '[3, -1, 8, 40]' is invalid for input of size 2560 File "/mnt/nvme/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/mnt/nvme/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/mnt/nvme/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/mnt/nvme/ComfyUI/nodes.py", line 1346, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) File "/mnt/nvme/ComfyUI/nodes.py", line 1316, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, File "/mnt/nvme/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 22, in informative_sample raise e File "/mnt/nvme/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 9, in informative_sample return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations. File "/mnt/nvme/ComfyUI/comfy/sample.py", line 43, in sample samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 761, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 663, in sample return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 650, in sample output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 629, in inner_sample samples = sampler.sample(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 534, in sample samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 870, in sample_unipc x = uni_pc.sample(noise, timesteps=timesteps, skip_type="time_uniform", method="multistep", order=order, lower_order_final=True, callback=callback, disable_pbar=disable) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 717, in sample model_prev_list = [self.model_fn(x, vec_t)] File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 410, in model_fn return self.data_prediction_fn(x, t) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 394, in data_prediction_fn noise = self.noise_prediction_fn(x, t) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 388, in noise_prediction_fn return self.model(x, t) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 329, in model_fn return noise_pred_fn(x, t_continuous) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 297, in noise_pred_fn output = model(x, t_input, **model_kwargs) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 861, in lambda input, sigma, **kwargs: predict_eps_sigma(model, input, sigma, **kwargs), File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 845, in predict_eps_sigma return (input - model(input, sigma_in, **kwargs)) / sigma File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 272, in __call__ out = self.inner_model(x, sigma, model_options=model_options, seed=seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 616, in __call__ return self.predict_noise(*args, **kwargs) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 619, in predict_noise return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 258, in sampling_function out = calc_cond_batch(model, conds, x, timestep, model_options) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 216, in calc_cond_batch output = model_options['model_function_wrapper'](model.apply_model, {"input": input_x, "timestep": timestep_, "c": c, "cond_or_uncond": cond_or_uncond}).chunk(batch_chunks) File "/mnt/nvme/ComfyUI/custom_nodes/comfyui-magic-clothing/attn_handler.py", line 142, in __call__ output = apply_model(input, timestep, **c) File "/mnt/nvme/ComfyUI/comfy/model_base.py", line 97, in apply_model model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float() File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/mnt/nvme/ComfyUI/custom_nodes/FreeU_Advanced/nodes.py", line 176, in __temp__forward h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator) File "/mnt/nvme/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 44, in forward_timestep_embed x = layer(x, context, transformer_options) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/mnt/nvme/ComfyUI/comfy/ldm/modules/attention.py", line 644, in forward x = block(x, context=context[i], transformer_options=transformer_options) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/mnt/nvme/ComfyUI/comfy/ldm/modules/attention.py", line 568, in forward n = attn2_replace_patch[block_attn2](n, context_attn2, value_attn2, extra_options) File "/mnt/nvme/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/CrossAttentionPatch.py", line 26, in __call__ out = out + callback(out, q, k, v, extra_options, **self.kwargs[i]) File "/mnt/nvme/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/CrossAttentionPatch.py", line 154, in ipadapter_attention out_ip = optimized_attention(q, ip_k, ip_v, extra_options["n_heads"]) File "/mnt/nvme/ComfyUI/comfy/ldm/modules/attention.py", line 352, in attention_pytorch q, k, v = map( File "/mnt/nvme/ComfyUI/comfy/ldm/modules/attention.py", line 353, in lambda t: t.view(b, -1, heads, dim_head).transpose(1, 2), ```
Error log 2 ``` Error occurred when executing KSampler: Sizes of tensors must match except in dimension 1. Expected size 3 but got size 9 for tensor number 1 in the list. File "/mnt/nvme/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/mnt/nvme/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/mnt/nvme/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/mnt/nvme/ComfyUI/nodes.py", line 1346, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) File "/mnt/nvme/ComfyUI/nodes.py", line 1316, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, File "/mnt/nvme/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 22, in informative_sample raise e File "/mnt/nvme/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 9, in informative_sample return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations. File "/mnt/nvme/ComfyUI/comfy/sample.py", line 43, in sample samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 761, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 663, in sample return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 650, in sample output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 629, in inner_sample samples = sampler.sample(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 534, in sample samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 870, in sample_unipc x = uni_pc.sample(noise, timesteps=timesteps, skip_type="time_uniform", method="multistep", order=order, lower_order_final=True, callback=callback, disable_pbar=disable) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 717, in sample model_prev_list = [self.model_fn(x, vec_t)] File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 410, in model_fn return self.data_prediction_fn(x, t) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 394, in data_prediction_fn noise = self.noise_prediction_fn(x, t) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 388, in noise_prediction_fn return self.model(x, t) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 329, in model_fn return noise_pred_fn(x, t_continuous) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 297, in noise_pred_fn output = model(x, t_input, **model_kwargs) File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 861, in lambda input, sigma, **kwargs: predict_eps_sigma(model, input, sigma, **kwargs), File "/mnt/nvme/ComfyUI/comfy/extra_samplers/uni_pc.py", line 845, in predict_eps_sigma return (input - model(input, sigma_in, **kwargs)) / sigma File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 272, in __call__ out = self.inner_model(x, sigma, model_options=model_options, seed=seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 616, in __call__ return self.predict_noise(*args, **kwargs) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 619, in predict_noise return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 258, in sampling_function out = calc_cond_batch(model, conds, x, timestep, model_options) File "/mnt/nvme/ComfyUI/comfy/samplers.py", line 216, in calc_cond_batch output = model_options['model_function_wrapper'](model.apply_model, {"input": input_x, "timestep": timestep_, "c": c, "cond_or_uncond": cond_or_uncond}).chunk(batch_chunks) File "/mnt/nvme/ComfyUI/custom_nodes/comfyui-magic-clothing/attn_handler.py", line 142, in __call__ output = apply_model(input, timestep, **c) File "/mnt/nvme/ComfyUI/comfy/model_base.py", line 97, in apply_model model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float() File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/mnt/nvme/ComfyUI/custom_nodes/FreeU_Advanced/nodes.py", line 176, in __temp__forward h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator) File "/mnt/nvme/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 44, in forward_timestep_embed x = layer(x, context, transformer_options) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/mnt/nvme/ComfyUI/comfy/ldm/modules/attention.py", line 644, in forward x = block(x, context=context[i], transformer_options=transformer_options) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/mnt/nvme/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/mnt/nvme/ComfyUI/comfy/ldm/modules/attention.py", line 510, in forward n, context_attn1, value_attn1 = p(n, context_attn1, value_attn1, extra_options) File "/mnt/nvme/ComfyUI/custom_nodes/comfyui-magic-clothing/attn_handler.py", line 246, in __call__ q = torch.cat([q, feature_hidden_states], dim=1) ```
longgui0318 commented 4 weeks ago

I will take care of it these few days.