是用zero2进行pt时，出现ValueError: `--mixed_precision` arg cannot be set to `bf16` when `fp16` is set in the DeepSpeed config file.

shibing624 / MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型，实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。

Apache License 2.0

3.24k stars 492 forks source link

是用zero2进行pt时，出现ValueError: `--mixed_precision` arg cannot be set to `bf16` when `fp16` is set in the DeepSpeed config file. #288

Closed erichuazhou closed 9 months ago

erichuazhou commented 9 months ago

Describe the Question

Please provide a clear and concise description of what the question is. 是用zero2进行pt时，出现ValueError: --mixed_precision arg cannot be set to bf16 when fp16 is set in the DeepSpeed config file.

请问大捞，应该如何正确设置？谢谢。

@shibing624

shibing624 commented 9 months ago

提示不是写了吗，deepspeed的config改下，要么都用bf16.