search

keras-team / keras-hub

Pretrained model hub for Keras 3
Apache License 2.0
765 stars 233 forks source link

Conversion script modification for Gemma 2 models #1691

Open sparsh35 opened 2 months ago

sparsh35 commented 2 months ago

Is your feature request related to a problem? Please describe. I am trying to convert a keras Gemma model after training to HF trasnformers , so that it can be used in VLLM , for faster inference. There is a script which can convert Gemma model to transformers at tools/gemma/export_gemma_to_hf.py but it is outdated.

Describe the solution you'd like If someone can modify it for Gemma2 models. And also write a script for LLAMA 3 models as I suppose these two are majorly used, and there is not a faster continous batching alternative in keras nlp ecosystem.

Describe alternatives you've considered

Additional context

Here is the script https://github.com/keras-team/keras-nlp/blob/master/tools/gemma/export_gemma_to_hf.py

mattdangerw commented 2 months ago

Thanks! Agreed on both counts.

If you'd like to go in an update the script, you are more then welcome. If not, we will get to this soon.