search

google / gemma.cpp

lightweight, standalone C++ inference engine for Google's Gemma models.
Apache License 2.0
5.88k stars 498 forks source link

Convert recurrentgemma weights #301

Closed 0wwafa closed 2 weeks ago

0wwafa commented 1 month ago

If I download google/recurrentgemma-2b-it from HF, I get a directory with safetensors. How do I convert it to the compressed file gemma.cpp wants? How do I run an interference using gemma.cpp with that particular model? (don't tell me to download the SBS from kaggle) My question is about: google/recurrentgemma-2b-it and the similar 9b directory

jan-wassenberg commented 1 month ago

@fischbacher implemented conversion scripts from RecurrentGemma to which are awaiting open-sourcing.

Downloading from Kaggle is currently the easiest/quickest path, I'm curious why that is not preferred?

0wwafa commented 1 month ago

@fischbacher implemented conversion scripts from RecurrentGemma to which are awaiting open-sourcing.

Downloading from Kaggle is currently the easiest/quickest path, I'm curious why that is not preferred?

because I already downloaded the model and I am on a metered connection ;)

jan-wassenberg commented 1 month ago

Ah, that makes sense :) Hopefully the scripts can be open sourced soon.