search

THUDM / ChatGLM-6B

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型
Apache License 2.0
40.47k stars 5.19k forks source link

如何在领域数据上二次预训练 #1167

Open ZTurboX opened 1 year ago

ZTurboX commented 1 year ago

Is your feature request related to a problem? Please describe.

No response

Solutions

如何在领域数据上二次预训练

Additional context

No response

MikeHollyWong commented 1 year ago

我也在做增量预训练，欢迎做类似的伙伴交流

feisan commented 1 year ago

正准备做这个事情，大家有什么推荐的方案吗？

ghost commented 1 year ago

插眼

greatheart1000 commented 1 year ago

现在有结果吗

SolarKnight1 commented 1 year ago

cy

feisan commented 1 year ago

看来官方暂时不太打算回答这方面的问题。

我准备按照这个项目来搞一搞 https://github.com/shibing624/MedicalGPT

tomcat123a commented 1 year ago

https://github.com/shibing624/MedicalGPT 参考这个项目，预训练，指令微调，rm模型训练，ppo都有现成的