search

THUDM / ChatGLM-6B

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型
Apache License 2.0
40.47k stars 5.19k forks source link

全量调参或者二次预训练怎么做？ #1272

Open xiaohengDa opened 1 year ago

xiaohengDa commented 1 year ago

Is there an existing issue for this?

Current Behavior

ptuning对于想让模型掌握特定领域知识的表现不太好，怎么根据数据集进行全量调参或者二次预训练呢？

Expected Behavior

No response

Steps To Reproduce

无

Environment

- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :

Anything else?

No response

guoqiangqi commented 1 year ago

同样想问

tomcat123a commented 1 year ago

https://github.com/shibing624/MedicalGPT 参考这个项目，预训练，指令微调，rm模型训练，ppo都有现成的。