hpcaitech / ColossalAI

Making large AI models cheaper, faster and more accessible
https://www.colossalai.org
Apache License 2.0
38.32k stars 4.3k forks source link

[FEATURE]: add parts for unsupervised learning #3570

Open wanghetongtt opened 1 year ago

wanghetongtt commented 1 year ago

Describe the feature

Supposed we want a "professional language model" for specific industry (say maritime transportation). Is there a way to train large amont of text data (say papers, books, newspaper) not in the format of Q&A form?

binmakeswell commented 1 year ago

Hi @wanghetongtt Yes, this can be done. However, this is currently beyond the scope of support from the open source community.

Feel free to contact me via email ybl@hpcaitech.com to discuss formal collaboration, and you will receive professional high-priority support to help you get your product development done quickly and at low cost. Thanks.