Closed ahuguenard-logility closed 1 month ago
So Google engineers found a mini bug which isn't public yet on this issue - see https://huggingface.co/google/gemma-2-9b/discussions/22
I added a PR into llama.cpp: https://github.com/ggerganov/llama.cpp/pull/8444
It works now! Thanks for the help.
Hi @danielhanchen , I tried training a gemma2 9b model today, but I ran into an error within llama.cpp when the model was converting from bf16 to f16. The problem arose due to the changes made in the config.json file:
This is the error from llama.cpp and the relevant code:![image](https://github.com/unslothai/unsloth/assets/170015693/c2cef631-7129-4099-b097-1834996d48d6)
Is this a valid check for llama.cpp to be doing? And if so, which of the three values in this equation needs to be updated in the config.json?
Thanks!!