search

shibing624 / MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型，实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。
Apache License 2.0
3.37k stars 507 forks source link

请问支持llama 3.1 微调？ #413

Open cqray1990 opened 2 months ago

cqray1990 commented 2 months ago

加载Meta-Llama-3.1-8B 做pretaining时报错 raise ValueError( ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as pad_token (tokenizer.pad_token = tokenizer.eos_token e.g.) or add a new pad token via tokenizer.add_special_tokens({'pad_token': '[PAD]'}).

shibing624 commented 2 months ago

我兼容下，加到todo

cqray1990 commented 2 months ago

我兼容下，加到todo

pretain 看的有支持llama的，但是微调llama3也不行，同样的错误 @shibing624