RuntimeError: "nll_loss_out_frame" not implemented for 'Half'

shibing624 / MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型，实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。

Apache License 2.0

2.94k stars 451 forks source link

RuntimeError: "nll_loss_out_frame" not implemented for 'Half' #387

Open Li-Jicheng opened 6 days ago

Li-Jicheng commented 6 days ago

跑https://colab.research.google.com/github/shibing624/MedicalGPT/blob/main/run_training_dpo_pipeline.ipynb#scrollTo=J5kYehpzESyt （run_training_dpo_pipeline.ipynb）这个脚本时，pretrain阶段报错

shibing624 commented 6 days ago

去掉--fp16, 设置 --torch_dtype auto

Li-Jicheng commented 6 days ago

去掉--fp16, 设置 --torch_dtype auto

感谢回复，操作后依然报错