On the Ckpt of Mu-Llama

[nicolaus625]

How to use your ckpt in the huggingface page? You have three pth file: checkpoint.pth, 7B.pth and LLaMA/7B/consolidated.00.pth

[crypto-code]

The checkpoint files provided are as follows:

.
├── ...
├── MU-LLaMA                
│   ├── ckpts
│   │   │── LLaMA                        -> Folder containing LLaMA-2 model weights and tokenizer
│   │   │   │── 7B                       -> Folder containing LLaMA-7B weights
│   │   │   │   │── checklist.chk
│   │   │   │   │── consolidated.00.pth  -> LLaMA-7B checkpoint
│   │   │   │   │── params.json
│   │   │   │── llama.sh
│   │   │   │── tokenizer.model
│   │   │   │── tokenizer_checklist.chk
│   │   │── 7B.pth                        -> Checkpoint for the LLaMA-Adapter model for evaluation
│   │   ├── checkpoint.pth                -> Fine-tuned Checkpoint for the MU-LLaMA model
└── ...

During training, we provide arguments to the folder containing the LLaMA weights (./ckpts/LLaMA) and specify the type as 7B so the weights are loaded from the 7B folder within the LLaMA folder specified.

During inference and evaluation of our model, we provide arguments to the folder containing the LLaMA weights (./ckpts/LLaMA) as well as the trained checkpoint for our MU-LLaMA model (./ckpts/checkpoint.pth).

For evaluation of the LLaMA Adapter model for comparison, we provide arguments to the folder containing the LLaMA weights (./ckpts/LLaMA) as well as the trained checkpoint for the LLaMA Adapter weights (./ckpts/7B.pth).

I hope this answers your question 😄