search

OptimalScale / LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
https://optimalscale.github.io/LMFlow/
Apache License 2.0
8.23k stars 823 forks source link

Support Mistral 7B model #652

Open wenlinyao opened 1 year ago

wenlinyao commented 1 year ago

Do you have the plan to support Mistral 7B model since it outperforms both LLaMA-1 and LLaMA-2? (https://mistral.ai/news/announcing-mistral-7b/) Thanks!

shizhediao commented 1 year ago

Hi, Thanks for your attention! Mistral is indeed a better model given their reported results. If you want to use it, you can simply change the model_name_or_path to the name of the checkpoint. For example, mistralai/Mistral-7B-v0.1. If you have any questions, feel free to let me know~

yiyiwwang commented 5 months ago

Hi, Thanks for your attention! Mistral is indeed a better model given their reported results. If you want to use it, you can simply change the model_name_or_path to the name of the checkpoint. For example, mistralai/Mistral-7B-v0.1. If you have any questions, feel free to let me know~

I tested Mistral-7B-v0.1, the version peft==0.4.0 in current environment of LMFlow would cause the problem "ValueError: Please specify target_modules in peft_config".

Update the peft version to the latest 0.10.1.dev0 would solve the problem, and the code can run.

wheresmyhair commented 5 months ago

Hi, Thanks for your attention! Mistral is indeed a better model given their reported results. If you want to use it, you can simply change the model_name_or_path to the name of the checkpoint. For example, mistralai/Mistral-7B-v0.1. If you have any questions, feel free to let me know~

I tested Mistral-7B-v0.1, the version peft==0.4.0 in current environment of LMFlow would cause the problem "ValueError: Please specify target_modules in peft_config".

Update the peft version to the latest 0.10.1.dev0 would solve the problem, and the code can run.

Thanks for your feedback and we just updated our requirements ❤️