search

kijai / ComfyUI-CogVideoXWrapper

815 stars 47 forks source link

feature request - CogVideoX1.5-5B-SAT, master kijai! #206

Open whmc76 opened 4 days ago

whmc76 commented 4 days ago

https://huggingface.co/THUDM/CogVideoX1.5-5B-SAT/tree/main

Minamiyama commented 4 days ago

go go go!

jpgallegoar commented 4 days ago

Nothing to do until they release diffusers version

kijai commented 4 days ago

Diffusers implementation of the new rotary embeds is not done, so the model doesn't run yet. Gotta wait a bit.

ruSauron commented 3 days ago

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

kijai commented 3 days ago

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

ruSauron commented 3 days ago

Sorry, I'm trying after node update to use the 1.5 models, but they both (T2V, I2V) don't work. With I2V I get the following error:

  • Exception Message: The size of tensor a (4050) must match the size of tensor b (2700) at non-singleton dimension 2

I've used ComfyUI-CogVideoXWrapper\examples\cogvideox_I2V_example_01.json as workflow, changed model to "kijai/CogVideoX-5b-1.5-I2V" and CLIP to "t5xxl_fp8_e4m3fn.safetensors"

I've also tried using other workflows and creating workflows from scratch.

git checkout
Your branch is up to date with 'origin/1.5_test'.
git rev-list --min-parents=2 --max-count=1 HEAD
21025c4742e16f43f33c99db9193d53f818797c9

Log: 15_error.txt

kijai commented 3 days ago

Sorry, I'm trying after node update to use the 1.5 models, but they both (T2V, I2V) don't work. With I2V I get the following error:

  • Exception Message: The size of tensor a (4050) must match the size of tensor b (2700) at non-singleton dimension 2

I've used ComfyUI-CogVideoXWrapper\examples\cogvideox_I2V_example_01.json as workflow, changed model to "kijai/CogVideoX-5b-1.5-I2V" and CLIP to "t5xxl_fp8_e4m3fn.safetensors"

I've also tried using other workflows and creating workflows from scratch.

git checkout
Your branch is up to date with 'origin/1.5_test'.
git rev-list --min-parents=2 --max-count=1 HEAD
21025c4742e16f43f33c99db9193d53f818797c9

Log: 15_error.txt

How many frames did you try with?

Seems odd, I followed those exact same steps now and it works for me.

ruSauron commented 3 days ago

16 frames, now tried with 17+ frames. Thanks, the cause of the problem was not obvious

kijai commented 3 days ago

16 frames, now tried with 17+ frames. Thanks, the cause of the problem was not obvious

Yeah it's weird with that, will make it more obvious once I'm certain what frame lengths work. Another thing I've noticed that the old default resolution (720x480) looks terrible, but for example 768x768 is nice. I think there's still something wrong in the current diffusers implementation.

littleyeson commented 16 hours ago

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

How many VRAM can be use for this 5b-1.5 Model?

kijai commented 16 hours ago

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

How many VRAM can be use for this 5b-1.5 Model?

Depends on frame count and resolution used, it's no different from before.