search

city96 / ComfyUI-GGUF

GGUF Quantization support for native ComfyUI models
Apache License 2.0
706 stars 40 forks source link

[Feature Request] Modify save checkpoint node #12

Open 311-code opened 1 month ago

311-code commented 1 month ago

Not sure if this feasable but would love to see a modified save checkpoint node for this somehow so we can save save gguf or exl2 for merging in loras to a gguf or exl2 checkpoint directly.

Check out nodes_model_merging.py def save_checkpoint and class checkpointsave. You can import flux related areas of comfyui or gguf related things.

Currently I am messing around with some of the if and elif stuff to bypass some of it and match similar keys from models of other architectures with matching state_dict layers names, by matching names at the end of matching prefix. You can also modify the nodes_model_merging_model_specifc.py to do whatever you want for any models more a more realtime view of changes with modified merging nodes instead of using disk space.

I have had success modiying these for forcing specific .weight layers of models in comfyui for models that are specified to not be able to be merged so I imaging gguf merging with loras may not be too difficult? (But not sure)

I dont know too much about creating gguf or exl2 for image generations yet, and only really use exl2 when doing llm things in oobabooga. Let me know what you think.

city96 commented 1 month ago

I think a save node for at least the current quants would be pretty simple, although I'm hesitant to make quantizing easy until the format/metadata/key names are finalized. I want to avoid having a lot of broken quants around that people would have to remake like the GGML->GGUF switch with LLMs. Hope that makes sense.