search

allenai / open-instruct

Apache License 2.0
1.08k stars 140 forks source link

Resize token embeddings of the base model before merging #142

Closed notoookay closed 1 month ago

notoookay commented 3 months ago

Hi, After I trained llama2-7b model with lora, I tried to merge it, it seems like resizing token embeddings of the base model should before merging, I reordered the process, and it worked. Can you check the correctness?

hamishivi commented 1 month ago

Thanks, this seems to work fine.