search

ContextualAI / gritlm

Generative Representational Instruction Tuning
https://arxiv.org/abs/2402.09906
MIT License
479 stars 33 forks source link

model before fine-tuning issue #12

Closed BlackHandsomeLee closed 3 months ago

BlackHandsomeLee commented 3 months ago

Hello author, I want to reproduce your paper. Is it okay to use Mistral-7B-Instruct-v0.2 for the model before fine-tuning?

BlackHandsomeLee commented 3 months ago

or maybe mistralai/Mistral-7B-v0.1 can be used too？

Muennighoff commented 3 months ago

Yeah you can do that but note that we used https://huggingface.co/mistralai/Mistral-7B-v0.1 If you use the Instruct ones you may want to change the chat format in the code to continue using theirs

BlackHandsomeLee commented 3 months ago

Yeah you can do that but note that we used https://huggingface.co/mistralai/Mistral-7B-v0.1 If you use the Instruct ones you may want to change the chat format in the code to continue using theirs

thank your for reply