search

OpenBMB / ModelCenter

Efficient, Low-Resource, Distributed transformer implementation based on BMTrain
https://modelcenter.readthedocs.io
Apache License 2.0
243 stars 30 forks source link

How can I use my own dataset while using ModelCenter？ #38

Open lhj-git opened 1 year ago

Achazwl commented 1 year ago

For finetune, refer to https://github.com/OpenBMB/ModelCenter/tree/main/model_center/dataset/t5dataset. For pretrain, refer to https://modelcenter.readthedocs.io/en/latest/notes/pretrain_data.html.