[BUG/Help] 统一回复，很多人要预训练，增量训练，多轮对话，ppo的代码

[tomcat123a]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

https://github.com/shibing624/MedicalGPT 参考这个项目，预训练，指令微调，rm模型训练，ppo都有现成的。

Expected Behavior

No response

Steps To Reproduce

https://github.com/shibing624/MedicalGPT 参考这个项目，预训练，指令微调，rm模型训练，ppo都有现成的。

Environment

- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :

Anything else?

No response

[tomcat123a]

需要交流加我微信 yongkangzhou

[cxjtju]

Is there an existing issue for this?

• [x] I have searched the existing issues

Current Behavior

https://github.com/shibing624/MedicalGPT 参考这个项目，预训练，指令微调，rm模型训练，ppo都有现成的。

Expected Behavior

No response

Steps To Reproduce

https://github.com/shibing624/MedicalGPT 参考这个项目，预训练，指令微调，rm模型训练，ppo都有现成的。

Environment

- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :

Anything else?

No response

请问预训练阶段训练文本最大长度是不能超过2048个token吗？基于MedicalGPT，超过2048会自动截断吗？报错ValueError: 130004 is not in list

[guoqiangqi]

需要交流加我微信 yongkangzhou

我们在做chatGLM-6B相关训练，希望和您保持交流，微信已申请好友 ：）