Open Azure-Tang opened 1 month ago
There usually are other formats with better perplexity and lower file size than Q4_0 and Q5_0. Do you really need them?
I have not heard back from you, but I have now implemented Q4_0 and Q5_0 anyway. Could you test whether it works for your use case?
sorry i didn't check this message box, I would test it in a few days and give you feedback. Thanks!
Should be fixed in https://github.com/99991/pygguf/pull/4
Should be fixed in #4
What does that pull request have to do with this issue?
At the end of the day (not literally), gguf package will handle dequantization, so all quantization type will be supported.
Hi, I notice some dequant format, like Q4_0 Q5_0, are not supported yet. Will you support these formats?