Open GameOverFlowChart opened 4 months ago
Can't you already create iquants by providing the right name? https://github.com/ggerganov/llama.cpp/blob/04976db7a819fcf8bfefbfc09a3344210b79dd27/gguf-py/gguf/constants.py#L811
Oh right in this case this should be added to the list of the names that is shown in the notebook, at least one of them as an example (so that you see at first glance that there is no underscore between I and Q.
Would it be possible to support i-Quants in AutoQuant or are they more demanding to quantize?