gemma.cc:1322: Failed to load model weight

[ordentid]

Hello, just fresh running the program after building for 8bit

./gemma \
--tokenizer tokenizer.spm \
--compressed_weights 2b-it-sfp.sbs \
--model 2b-it

and it output

Cached compressed weights does not exist yet (code 227), compressing weights and creating file: 2b-it-sfp.sbs.
Abort at ~/gemma.cpp/gemma.cc:1322: Failed to load model weights.
[1]    6581 abort      ./gemma --tokenizer tokenizer.spm --compressed_weights 2b-it-sfp.sbs --model

any idea what is wrong?

[jan-wassenberg]

This appears to be an older version of the code :) Does it help to update to git HEAD? FYI we have just merged dev branch into main, so either is fine.

[alankelly]

I have the same issue running from HEAD.

./gemma --tokenizer /tmp/gemma_cpp_weights/tokenizer.spm --weights /tmp/gemma_cpp_weights/2b-pt-mqa.sbs --model 2b-pt

Cached compressed weights does not exist yet (code 104), compressing weights and creating file: /tmp/gemma_cpp_weights/2b-pt-mqa.sbs. Abort at /usr/local/google/home/alankelly/dev/gemma.cpp/./gemma/gemma.cc:1394: Failed to load model weights. Aborted

[jan-wassenberg]

Hi @alankelly , the 'code' in the error message is a line number. Looks like the .sbs file has a size that does not match the size in the header. Might it be an incomplete download? Did you get the latest version of the weights from Kaggle?

A while back, there was an incompatible change to the tokenizer vocab, and weights had to be updated. I am not 100% sure all downloads on Kaggle were successfully updated. Maybe also try the IT instruction tuned version?

[alankelly]

Thanks @jan-wassenberg this works now after re-downloading the weights

[jan-wassenberg]

Nice, thanks for following up :)

[KumarGitesh2024]

Hi @alankelly Thank you for the confirmation on the resolved issues, Hence closing the issue.