search

Kosinkadink / ComfyUI-AnimateDiff-Evolved

Improved AnimateDiff for ComfyUI and Advanced Sampling Support
Apache License 2.0
2.76k stars 208 forks source link

FreeInit Error- Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! #452

Open kendinitto opened 3 months ago

kendinitto commented 3 months ago

Getting the following error when using FreeInit Iteration Options, if I do not use FreeInit everything works perfectly. Everything else in ComfyUI is working normally except this one thing. It used to work but after updating to the latest it no longer works.

[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (32) greater than context_length 16. [AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2. [AnimateDiffEvo] - INFO - Iteration 1/2 !!! Exception during processing!!! Traceback (most recent call last): File "I:\AI\ComfyUI\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack\inspire\a1111_compat.py", line 229, in sample latent = inspire_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise, noise_mode, incremental_seed_mode=batch_seed_mode, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack\inspire\a1111_compat.py", line 96, in inspire_ksampler raise e File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack\inspire\a1111_compat.py", line 84, in inspire_ksampler samples = common.impact_sampling( ^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack\inspire\libs\common.py", line 14, in impact_sampling return nodes.NODE_CLASS_MAPPINGS['RegionalSampler'].separated_sample(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\special_samplers.py", line 408, in separated_sample return separated_sample(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_sampling.py", line 214, in separated_sample res = sample_with_custom_noise(model, add_noise, seed, cfg, positive, negative, impact_sampler, sigmas, latent_image, noise=noise, callback=callback) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_sampling.py", line 158, in sample_with_custom_noise samples = comfy.sample.sample_custom(model, noise, cfg, sampler, sigmas, positive, negative, latent_image, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 536, in motion_sample latents, noise = iter_opts.preprocess_latents(curr_i=curr_i, model=model, latents=latents, noise=noise, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sample_settings.py", line 475, in preprocess_latents noised_latents = latents sqrt_alpha_prod + noise sqrt_one_minus_alpha_prod


RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Prompt executed in 2.48 seconds

![freeinit1](https://github.com/user-attachments/assets/46303d34-04dc-4a28-819d-1972df1297e5)
![freeinit2](https://github.com/user-attachments/assets/50bd2741-7eaf-4e9f-84a2-b1b910673724)
Kosinkadink commented 3 months ago

Thank you for the report, I'll look into it

kendinitto commented 3 months ago

BTW, I am running with an RTX 4090 and 32GB of internal Ram so I got plenty of juice.

Kosinkadink commented 2 months ago

I've been unable to replicate your issue, could you create the simplest workflow you can that has the problem? And can you try to use the vanilla KSamplers from ComfyUI? (you can start off with this txt2vid example in the readme: https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved?tab=readme-ov-file#txt2vid)

kendinitto commented 2 months ago

Could it be related to my torch version?

Total VRAM 24564 MB, total RAM 32583 MB pytorch version: 2.4.0+cu121 Set vram state to: NORMAL_VRAM Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync Using pytorch cross attention`

The only other time i've ever seen an error similar to that in the past , it was related to torch

Kosinkadink commented 2 months ago

Did you try it with a normal ksampler?

kendinitto commented 2 months ago

Ok so I just tried the normal ksampler and that does in fact work, but when I use KSamplerPipe //Inspire then I get the errors so the problem seems related to that node.

Kosinkadink commented 2 months ago

Cool, sometime I'll try out that specific node that gives you trouble and I'll see if there's a workaround I can add in my code to just make FreeInit work with it. Something is just not on the right device when it gets there when using that other KSampler.

Kosinkadink commented 2 months ago

But for now you can just use other KSamplers to workaround it

kendinitto commented 2 months ago

I just did git pull of the latest inspire pack and get the same error

kendinitto commented 2 months ago

Ya ill have to switch ksampler to get around it thanks.

kendinitto commented 2 months ago

One more update , I tried ksampler pipe from impakt as well as non pipe versions and none of them work, I also did not see a way to get the AYS SD1 scheduler option with the standard working ksampler which I have found that one to be noticeably the best scheduler to use for animatediff.

kendinitto commented 2 months ago

FYI, as another work around Ksampler from efficient does work and that has AYS SD1