search

young-geng / EasyLM

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.
Apache License 2.0
2.33k stars 247 forks source link

Mistral #104

Open peregilk opened 8 months ago

peregilk commented 8 months ago

Are the any plans for adding support for Mistral-training? What would be required for this?

simonucl commented 6 months ago

I have forked the repository and added support for Mistral implementation in my forked branch. Most of the code in Llama implementation are reused since they have nearly the same architecture. The training code for Mistral will be made available soon. I hope this helps, and any discussion is welcome.

peregilk commented 6 months ago

Hi @simonucl. Thanks for letting me know. Looked through your code, and it seems great.

Could you contact me regarding the status of the training script? Maybe we can to something together here.

simonucl commented 6 months ago

Sent an email to you!

R4ZZ3 commented 6 months ago

Any news on this?

versae commented 3 months ago

@simonucl, @peregilk any luck with the training script?