Open GamingDaveUk opened 1 day ago
Did the patch apply successfully? That's the default error when you try to use the base llama.cpp llama-quantize
binary without the patch applied iirc.
no there was a crc error on the patch, i assumed that it meant the patch was already in the main code
I have llama.cpp installed in its own instance so it was a pain to follow the instruction, i may have messed up a step.
I will try again tomorrow when more awake.
Okay yeah, that's probably the problem then. The actual upstream repo isn't meant for image models; the patch is the part that adds support for quantizing flux.
If you post the actual error where the patch apply fails I might be able to help out.
(It could just be this as well, i.e. line ending mismatch due to git converting them when cloning: https://github.com/city96/ComfyUI-GGUF/issues/90#issuecomment-2323011648 )
Trying to quantise some flux models to lower the vram needs and I get that error.
Is flux not supported for quantisastion?