-
how convert microsoft/Phi-3.5-MoE-instruct to gguf?
-
### Feature Idea
I have tested the quantization of gguf on SD3 and Flux, and the results are great, smaller memory occupancy and faster speed, hope to support
### Existing Solutions
https://github.…
-
How can I set the weight_dtype as GGFU? Because it doesn't appeared in the list.
![image](https://github.com/user-attachments/assets/4f5150d5-0483-4f9b-846f-801907baab97)
-
Is there an easy way to convert gguf to marlin and vice-versa? Any comparisons?
https://github.com/leafspark/AutoGGUF
-
Any chance 2bit models can be used with llama.cpp? Would be great to get LLama 3.1 (8B and 70B) converted to GGUF to try them out locally.
Thanks for the great research work!
-
I'm trying to run local llm with this model, where can I find the gguf file of this model on huggingface?
-
do you planing to do this for diffusers ,,, here the setup that made for bnb-NF4
https://github.com/huggingface/diffusers/issues/9165
i think you can remove bnb-nf4 stuff and add gguf
its featur…
-
Hi master Kijai, did your wrapper support this Kolors gguf model.
https://huggingface.co/twodgirl/Kolors-unet-gguf
-
It would be great to see OLMoE/OlmoeForCausalLM Llama.cpp/GGUF support.
Really neat project!
-
[GGUF](https://huggingface.co/docs/hub/en/gguf) is becoming a preferred means of distribution of FLUX fine-tunes.
Transformers recently added general support for GGUF and are slowly adding support …