Gemma2 9B + Unsloth

[ahuguenard-logility]

Hi @danielhanchen , I tried training a gemma2 9b model today, but I ran into an error within llama.cpp when the model was converting from bf16 to f16. The problem arose due to the changes made in the config.json file:

This is the error from llama.cpp and the relevant code:

Is this a valid check for llama.cpp to be doing? And if so, which of the three values in this equation needs to be updated in the config.json?

Thanks!!

[danielhanchen]

So Google engineers found a mini bug which isn't public yet on this issue - see https://huggingface.co/google/gemma-2-9b/discussions/22

I added a PR into llama.cpp: https://github.com/ggerganov/llama.cpp/pull/8444

[ahuguenard-logility]

It works now! Thanks for the help.