Closed younesbelkada closed 7 months ago
Hi, @younesbelkada. We intend to prepare 2bit gemma models and will come back to you once we have results.
Thanks a lot @Godofnothing ! Looking forward to it !
Hi, @younesbelkada. We have uploaded 2b gemma versions to the huggingface hub:
For some reason, 7b model gemma experiences significant decline in performance, making it unusable. In case we manage to resolve the issue, we will upload 7b models as well.
Very nice thank you @Godofnothing !
Hi authors!
With the recent AQLM integration in transformers, would it makes sense to quantize the Google gemma models in 2-bit
The list of the models can be found here: https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b
cc @BlackSamorez