Failed to read from model.weights.h5 - might be a directory, or too small?

[charbull]

Hi,

I am experiencing the follow issue, I tried the following versions: https://www.kaggle.com/models/keras/gemma/frameworks/Keras/variations/gemma_2b_en/versions/1 https://www.kaggle.com/models/keras/gemma/frameworks/Keras/variations/gemma_2b_en/versions/2

/gemma \                                         
--tokenizer vocabulary.spm \
--weights model.weights.h5 \
--compressed_weights 2b-pt-sfp.sbs  --model 2b-pt --verbosity 2
Cached compressed weights does not exist yet (code 256), compressing weights and creating file: 2b-pt-sfp.sbs.
Abort at /Users/charbel/Downloads/gemma/gemma.cpp/./gemma.cc:138: Failed to read from model.weights.h5 - might be a directory, or too small?
zsh: abort      ./gemma --tokenizer vocabulary.spm --weights model.weights.h5  2b-pt-sfp.sbs 

Any ideas how to resolve?

Cheers, Charbel

[austinvhuang]

Hi, in general you shouldn't use the --weights parameter at this time.

In the future you can use it to load fine tuned weights and make compressed versions, but that requires a python script to convert weights https://github.com/google/gemma.cpp/issues/11

Instead, only use the compressed weights (you don't need the keras weights, just the sfp files from the GemmaCpp download page).

Second, you probably want to start with the -it "instruction tuned" models which are more appropriate for interactive use. the -pt "pretrained models" are more of a starting point for fine tuning. So in summary:

• Download 2b-it-sfp.sbs
• Run with: this tweak to your command (remove --weights, pt->it):

  ./gemma \                                         
  --tokenizer vocabulary.spm \
  --compressed_weights 2b-it-sfp.sbs  --model 2b-it

  and see if that works.

[austinvhuang]

Closing for now but if you still run into an issue we'll reopen and help.

[charbull]

Thanks ! that worked, I was using the keras weights.