kijai / ComfyUI-CogVideoXWrapper

981 stars 59 forks source link

'image_cond_start_percent' where it is not associated with a value #258

Closed fergomez1 closed 2 days ago

fergomez1 commented 2 days ago

Hi All, I downloaded the flow cogvideox_1_0_5b_T2V_02.json when I ran, it came out this error, and I can not find a way to resolve it. aa

KrakeyMTL commented 2 days ago

Did you make this node yourself or use an old one? the v8 breaks all old - every node value is off when it re-loads the new shell.

try to hit refresh on the right menu after fully updating to get the latest wrapper changes. not sure if you did this already, we need a bit more info thanks!

also i'm a bit jealous you are already working on the text to vid portion lol I'm just solving the i2v hehe

fergomez1 commented 2 days ago

Hi, thank you for your quick reply. I did a clean install and then from the examples directory I loaded the cogvideox_1_0_5b_T2V_02.json workflow and when I run it. I get that error. Hope this helps. Thanks

KrakeyMTL commented 2 days ago

There was another fellow that posted a few threads down about the json having something. I don't know?! maybe the folder one has something when downloaded.

Weirdly i see it 615KB in size?! the RAW json when saved into notepad is only 13KB ?!

For whatever reason, Master Kijai can check this tomorrow just in case haha.

I resaved the RAW here in case you want to try? I really don't know past this bud I have it working from the same file.

RAW_MANUAL_RESAVE_cogvideox_1_5_5b_I2V_01.json

KrakeyMTL commented 2 days ago

oups I'm very tired. very. off to bed for me.

I just realized you were asking for the text to video not image to video. so sorry!!!

But, if one works the others should work if you still want to take a shot at the img2vid using the json.

I'll get some rest and look at this again with a coffee haha.

MiloHX commented 2 days ago

@fergomez1 If you are very urgent, you can apply a temporary fix by editing the nodes.py and adding image_cond_start_percent = 0.0 image_cond_end_percent = 1.0 right before if image_cond_latents is not None: assert supports_image_conds, "Image condition latents only supported for I2V and Interpolation models"

Zhenyi-Wang commented 2 days ago

@fergomez1 If you are very urgent, you can apply a temporary fix by editing the nodes.py and adding image_cond_start_percent = 0.0 image_cond_end_percent = 1.0 right before if image_cond_latents is not None: assert supports_image_conds, "Image condition latents only supported for I2V and Interpolation models"

Had the same issue. With this fix it reports another error. Trace:

Sampling 1 frames in 1 latent frames at 360x240 with 50 inference steps
  0%|                                                                                                                                                                | 0/50 [00:01<?, ?it/s]!!! Exception during processing !!! The size of tensor a (45) must match the size of tensor b (44) at non-singleton dimension 4
Traceback (most recent call last):
  File "/home/zhenyi/test/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zhenyi/test/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zhenyi/test/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/zhenyi/test/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zhenyi/test/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper/nodes.py", line 697, in process
    height = height,
          ^^^^^^^^^^
  File "/home/zhenyi/miniconda3/envs/comfyui/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/zhenyi/test/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper/pipeline_cogvideox.py", line 780, in __call__
    latents = self.scheduler.step(noise_pred, t, latents.to(self.vae_dtype), **extra_step_kwargs, return_dict=False)[0]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zhenyi/miniconda3/envs/comfyui/lib/python3.11/site-packages/diffusers/schedulers/scheduling_ddim_cogvideox.py", line 383, in step
    pred_original_sample = (alpha_prod_t**0.5) * sample - (beta_prod_t**0.5) * model_output
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Zhenyi-Wang commented 2 days ago

Seems it was caused by changing frame size and qty.. But with default settings it will cause OOM with 32G RAM and 22G VRAM... I see a lot other people reporting memory related issues, how confuing it is for beginners...

kijai commented 2 days ago

Yeah this was silly mistake as I didn't take text2vid into account when adding those options, fixed.