search

Leeroo-AI / mergoo

A library for easily merging multiple LLM experts, and efficiently train the merged LLM.
https://www.leeroo.com/
GNU Lesser General Public License v3.0
397 stars 25 forks source link

It will be a good idea to support T5 series model. #12

Open svjack opened 4 months ago

svjack commented 4 months ago

Before llama come to our sight, Big Science release MT0 and T0, which are forerunners of llm, and at the end of 2022, some repos take T5-large and T5-3b T5-11b as their base to train their small llm. It seems that T5 series are more suitable to be the ability of Bert takes in this project. Are there any plans to support T5 ? 😊

gitsailor5 commented 4 months ago

Hey @svjack! We've planned for T5 after a few pending releases, but if there's more interest, we can fast-track it!
( Just two more T5 requests on this thread, and we push it )