How to use utils.extend_model_vocab?

CrispStrobe commented 4 months ago

thanks for providing this toolset. i am unsure how utils.extend_model_vocab is intended to be used. in current form, it takes checkpoints only. when i try to adapt it to extend a model in safetensor format like so, it outputs a sequence like [control_29][control_698][control_638]...

matheus-prandini commented 3 months ago

@CrispStrobe What are you trying to achieve? Do you want to add specific new tokens?

CrispStrobe commented 3 months ago

would it be possible to adapt an existing eg 7b mistral v0.1/0.2 based finetuned model so that it works with the new extended 32768 vocabulary and the v3 tokenizer, and so that we could merge it with v3 models?

kehuitt commented 2 months ago

@matheus-prandini Hi! I met the same problem. I merged the base model and lora by the script below

python utils/merge_lora.py \
    --initial_model_ckpt Mixtral-8x7B-v0.1-Instruct_extended/consolidated.00.pth \
    --lora_ckpt mistral_8x7B/snapshots/f7ef571615ca95187c38d1e5451758532cf795c3/checkpoint_000948/consolidated/lora.safetensors \
    --dump_ckpt Model/merged_model.safetensors

Then I converted its weights to hf, but when I inferred from this merged model, I got the same output like [control_29][control_698][control_638]...

I would like to know which step I am having problems with, is it the merge step that is the problem? Thanks!!

mistralai / mistral-finetune

How to use utils.extend_model_vocab? #12