search

kijai / ComfyUI-CogVideoXWrapper

1.02k stars 61 forks source link

feature request - CogVideoX1.5-5B-SAT, master kijai! #206

Closed whmc76 closed 6 days ago

whmc76 commented 2 weeks ago

https://huggingface.co/THUDM/CogVideoX1.5-5B-SAT/tree/main

Minamiyama commented 2 weeks ago

go go go!

jpgallegoar commented 2 weeks ago

Nothing to do until they release diffusers version

kijai commented 2 weeks ago

Diffusers implementation of the new rotary embeds is not done, so the model doesn't run yet. Gotta wait a bit.

ruSauron commented 2 weeks ago

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

kijai commented 2 weeks ago

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

ruSauron commented 2 weeks ago

Sorry, I'm trying after node update to use the 1.5 models, but they both (T2V, I2V) don't work. With I2V I get the following error:

  • Exception Message: The size of tensor a (4050) must match the size of tensor b (2700) at non-singleton dimension 2

I've used ComfyUI-CogVideoXWrapper\examples\cogvideox_I2V_example_01.json as workflow, changed model to "kijai/CogVideoX-5b-1.5-I2V" and CLIP to "t5xxl_fp8_e4m3fn.safetensors"

I've also tried using other workflows and creating workflows from scratch.

git checkout
Your branch is up to date with 'origin/1.5_test'.
git rev-list --min-parents=2 --max-count=1 HEAD
21025c4742e16f43f33c99db9193d53f818797c9

Log: 15_error.txt

kijai commented 2 weeks ago

Sorry, I'm trying after node update to use the 1.5 models, but they both (T2V, I2V) don't work. With I2V I get the following error:

  • Exception Message: The size of tensor a (4050) must match the size of tensor b (2700) at non-singleton dimension 2

I've used ComfyUI-CogVideoXWrapper\examples\cogvideox_I2V_example_01.json as workflow, changed model to "kijai/CogVideoX-5b-1.5-I2V" and CLIP to "t5xxl_fp8_e4m3fn.safetensors"

I've also tried using other workflows and creating workflows from scratch.

git checkout
Your branch is up to date with 'origin/1.5_test'.
git rev-list --min-parents=2 --max-count=1 HEAD
21025c4742e16f43f33c99db9193d53f818797c9

Log: 15_error.txt

How many frames did you try with?

Seems odd, I followed those exact same steps now and it works for me.

ruSauron commented 2 weeks ago

16 frames, now tried with 17+ frames. Thanks, the cause of the problem was not obvious

kijai commented 2 weeks ago

16 frames, now tried with 17+ frames. Thanks, the cause of the problem was not obvious

Yeah it's weird with that, will make it more obvious once I'm certain what frame lengths work. Another thing I've noticed that the old default resolution (720x480) looks terrible, but for example 768x768 is nice. I think there's still something wrong in the current diffusers implementation.

littleyeson commented 1 week ago

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

How many VRAM can be use for this 5b-1.5 Model?

kijai commented 1 week ago

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

How many VRAM can be use for this 5b-1.5 Model?

Depends on frame count and resolution used, it's no different from before.