Open etwoods opened 1 year ago
https://github.com/tencent-ailab/IP-Adapter the IP-Adapter has release an version for SDXL
I'm currently working on it.
I'm currently working on it.
Thanks a lot!
waiting for it! hopefully is not gonna need a lot of changes
I probably did it. ^^^^^^
I probably did it. ^^^^^^
great! i'll test it out immediately
I probably did it. ^^^^^^
any idea about this issue "
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
loaded straight to GPU
loading new
loading new
loading new
loading new
0% 0/30 [00:00<?, ?it/s]/usr/local/lib/python3.10/dist-packages/torchsde/_brownian/brownian_interval.py:594: UserWarning: Should have tb<=t1 but got tb=14.614643096923828 and t1=14.614643.
warnings.warn(f"Should have {tb_name}<=t1 but got {tb_name}={tb} and t1={self._end}.")
67% 20/30 [00:17<00:08, 1.18it/s]loading new
67% 20/30 [00:21<00:10, 1.07s/it]
!!! Exception during processing !!!
Traceback (most recent call last):
File "/content/drive/MyDrive/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/content/drive/MyDrive/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/content/drive/MyDrive/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(slice_dict(input_data_all, i)))
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI_Fooocus_KSampler/sampler/nodes.py", line 49, in sample
return (core.ksampler_with_refiner(model, positive, negative, refiner_model, refiner_positive, refiner_negative, latent_image, noise_seed, steps, refiner_switch_step, cfg, sampler_name, scheduler, denoise=denoise, disable_noise=disable_noise, start_step=start_at_step, last_step=end_at_step, force_full_denoise=force_full_denoise), )
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, *kwargs)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI_Fooocus_KSampler/sampler/Fooocus/core.py", line 243, in ksampler_with_refiner
samples = sampler.sample(noise, positive_copy, negative_copy, refiner_positive=refiner_positive_copy,
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI_Fooocus_KSampler/sampler/Fooocus/samplers_advanced.py", line 236, in sample
samples = getattr(k_diffusionsampling, "sample{}".format(self.sampler))(self.model_k, noise, sigmas,
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(args, kwargs)
File "/content/drive/MyDrive/ComfyUI/comfy/k_diffusion/sampling.py", line 701, in sample_dpmpp_2m_sde_gpu
return sample_dpmpp_2m_sde(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, eta=eta, s_noise=s_noise, noise_sampler=noise_sampler, solver_type=solver_type)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/content/drive/MyDrive/ComfyUI/comfy/k_diffusion/sampling.py", line 615, in sample_dpmpp_2m_sde
callback({'x': x, 'i': i, 'sigma': sigmas[i], 'sigma_hat': sigmas[i], 'denoised': denoised})
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI_Fooocus_KSampler/sampler/Fooocus/samplers_advanced.py", line 223, in
Prompt executed in 157.83 seconds
usually i have no issues of memory on 24 gb ram
with searge again, usually works on t4 with extra ram: making attention of type 'vanilla-xformers' with 512 in_channels building MemoryEfficientAttnBlock with 512 in_channels... Working with z of shape (1, 4, 32, 32) = 4096 dimensions. making attention of type 'vanilla-xformers' with 512 in_channels building MemoryEfficientAttnBlock with 512 in_channels... loading new 100% 24/24 [00:20<00:00, 1.15it/s] loading new !!! Exception during processing !!! Traceback (most recent call last): File "/content/drive/MyDrive/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/content/drive/MyDrive/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/content/drive/MyDrive/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(*slice_dict(input_data_all, i))) File "/content/drive/MyDrive/ComfyUI/custom_nodes/SeargeSDXL/modules/sampling.py", line 199, in sample return nodes.common_ksampler(refiner_model, noise_seed + noise_offset, steps, cfg, sampler_name, scheduler, refiner_positive, refiner_negative, base_result[0], denoise=denoise refiner_strength, disable_noise=False, start_step=base_steps, last_step=steps, force_full_denoise=True) File "/content/drive/MyDrive/ComfyUI/nodes.py", line 1176, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, File "/content/drive/MyDrive/ComfyUI/comfy/sample.py", line 81, in sample comfy.model_management.load_models_gpu([model] + models, comfy.model_management.batch_area_memory(noise.shape[0] noise.shape[2] noise.shape[3])) File "/content/drive/MyDrive/ComfyUI/comfy/model_management.py", line 368, in load_models_gpu cur_loaded_model = loaded_model.model_load(lowvram_model_memory) File "/content/drive/MyDrive/ComfyUI/comfy/model_management.py", line 259, in model_load raise e File "/content/drive/MyDrive/ComfyUI/comfy/model_management.py", line 255, in model_load self.real_model = self.model.patch_model(device_to=patch_model_to) #TODO: do something with loras and offloading to CPU File "/content/drive/MyDrive/ComfyUI/comfy/sd.py", line 404, in patch_model self.model.to(device_to) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1145, in to return self._apply(convert) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 797, in _apply module._apply(fn) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 797, in _apply module._apply(fn) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 797, in _apply module._apply(fn) [Previous line repeated 3 more times] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 820, in _apply param_applied = fn(param) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1143, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) torch.cuda.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory) Currently allocated : 14.52 GiB Requested : 81.00 MiB Device limit : 14.75 GiB Free (according to CUDA): 4.81 MiB PyTorch limit (set by user-supplied memory fraction) : 17179869184.00 GiB
Since clip_vision_g is quite large, could that be the cause?
Since clip_vision_g is quite large, could that be the cause?
could be but i feel a little stuck on this
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1664]). 这个问题有遇到吗大家,尺寸不匹配,我换了1.5和SDXL的都不行
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1664]). 这个问题有遇到吗大家,尺寸不匹配,我换了1.5和SDXL的都不行
did you use models for sdxl?
引发运行时错误('加载 {}:\n\t{} 的state_dict中的错误。format( 运行时错误:重新采样器加载state_dict时出错:proj_in的大小不匹配.weight:使用形状火炬复制参数。尺寸([768, 1280])从检查点开始,当前模型中的形状是割炬。大小([768, 1664]).这个问题有遇到吗大家,尺寸不匹配,我换了1.5和SDXL的都不行
您是否使用 SDXL 模型?
是我更新了不稳定的controlnet插件,导致的,后面就正常了,感谢回复
really awesome work thank guys for that
do you have a plan for supporting SDXL?