No checkpoint files found in llama2-7b-chat

[timfpark]

I'm attempting to try llama2-burn with the standard llama2 model downloaded from Facebook and I'm seeing the following error when running test.py:

$ python llama-py/test.py ../llama2/llama2-7b-chat ../llama2/tokenizer.model
#words: 32000 BOS ID: 1 EOS ID: 2 PAD ID: -1
#words: 32000 BOS ID: 1 EOS ID: 2 PAD ID: -1
An error occurred: No checkpoint files found in ../llama2/llama2-7b-chat

Where llama2 is the directory for the cloned https://github.com/facebookresearch/llama repo and which has had download.sh run on it.

Environment: Ubuntu 22.04 on Intel i9

[Gadersd]

Did you convert the the model files to burn's format?

[filopedraz]

Getting the same error @Gadersd

I downloaded the official weights from llama.

checkpoints = sorted(Path(model_dir).glob("*.safetensors"))
if len(checkpoints) == 0:
   raise ValueError(f"No checkpoint files found in {model_dir}")

But in the test.py script the load_model method searches for *.safetensors.

The Llama team probably compressed all the weights in a single file consolidated.00.pth

I created a PR (#7)

[Gadersd]

I had accidentally modified test.py to use safetensors. The issue is now resolved.