OpenBMB / BMTrain

Efficient Training (including pre-training and fine-tuning) for Big Models
Apache License 2.0
560 stars 77 forks source link

Model extensibility #86

Closed pilipala818 closed 1 year ago

pilipala818 commented 1 year ago

Thanks for your great work,

On the official website, BMTrain support the following structure: Encoder(bert-base-cased bert-base-uncased bert-large-cased bert-large-uncased bert-base-chinese bert-base-multilingual-cased) Decoder(CPM-1(large) GPT-2(base) GPT-2(medium) GPT-2(large) GPT-2(XL) GPT-J(6B)) Encoder-Decoder(CPM-2(large) T5-small T5-base T5-large T5(3B) T5(11B))

Whether BMTrain support other model that outside the list?(e.g. resnet and so on) Is there a tutorial?

Looking forward your prompt reply.

a710128 commented 1 year ago

The models in the list are ones that we have implemented and verified based on BMTrain. In fact BMTrain can support a wide variety of models.

pilipala818 commented 1 year ago

Thanks a lot!!!