Incompatibility of "ClipLoader (GGUF)" and "triton-windows" when working with CogVideoX/Mochi (colored cubes)

[Ratinod]

When generating video in ComfyUI-CogVideoXWrapper using "ClipLoader (GGUF)" the result is "colored cubes". If you use a non-GGUF version, everything works as expected. more details: https://github.com/kijai/ComfyUI-CogVideoXWrapper/issues/200 Maybe this can be fixed somehow?

[city96]

Huh. I can't think of anything in this node pack that would actually cause that, other than a precision issue somewhere where the T5 hidden states get forced to FP16 or inference runs in full FP16 (iirc that did look something like that, even with flux) but I don't know why swapping torch sdpa for flash attn (sage attention?) would mess with it.

Can you give some more info on what your setup looks like (specifically the sage attention/flash attention/triton windows parts). (would also be good to know if this is reproducible with flux or if it's specific to the linked video model).

[Ratinod]

my setup:

comfyui: https://github.com/comfyanonymous/ComfyUI/releases/tag/v0.2.6 ( 2.5.0+cu124 + Python version: 3.12.7)

python_embeded\python.exe -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124 -> PyTorch 2.5.1

triton 3.1: https://github.com/woct0rdho/triton-windows/releases/download/v3.0.0-windows.post1/python_3.12.7_include_libs.zip + https://github.com/woct0rdho/triton-windows/releases/download/v3.1.0-windows.post5/triton-3.1.0-cp312-cp312-win_amd64.whl Following the instructions (https://github.com/woct0rdho/triton-windows/?tab=readme-ov-file#install-from-wheel) is important (and msvcp140.dll, vcruntime140.dll).

python_embeded\python.exe -m pip install triton-3.1.0-cp312-cp312-win_amd64.whl python_embeded\python.exe -m pip install sageattention

other: ComfyUI-GGUF, ComfyUI-CogVideoXWrapper, ComfyUI-MochiWrapper , ComfyUI-Manager, (+ a few extras)

pytorch version: 2.5.1+cu124 + triton 3.1 + sageattention +"ClipLoader (GGUF)" -> colored cubes pytorch version: 2.5.1+cu124 + triton 3.1 + sageattention +"Load CLIP" -> works

---

Notes:

• I can't compile "flash attention" for 2.5.1+cu124, i.e. I can't check its functionality. (there are no ready-made .whl windows.)
• ComfyUI-MochiWrapper gives the same colored cubes https://github.com/kijai/ComfyUI-CogVideoXWrapper/issues/200#issuecomment-2453050240
• considering the console and compilation time, not everything that should be compiled is compiled (Initial compilation time with "Load CLIP" is longer than with "ClipLoader (GGUF)")
• "DualClipLoader (GGUF)" on flux does not produce colored squares. (not sure if "sage attention" works at all, the speed does not change with or without "sage attention")