shibing624 / MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型,实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。
Apache License 2.0
3.24k stars 492 forks source link

【数据集】Alpaca 和 Vicuna 两个 Template 之间有什么不同 #268

Closed SoYuCry closed 10 months ago

SoYuCry commented 10 months ago

Describe the Question

Please provide a clear and concise description of what the question is.

想请教一下,Alpaca 和 Vicuna 这两个模板在格式上有什么不一样吗

8c02ec49d8af69ef7649be418e021e9

我了解到的是:相较于Alpaca,Vicuna在训练中将序列长度由512扩展到了2048,并且数据是从 ShareGPT.com 来的,而 Alpaca 的数据是从 GPT 3.5 来的。

Alpaca 是 instruction,input,output Vicuna 是 User,assitant,User,assitant ...

一方面长度不同,另一方面是 Vicuna 没有 instruction 部分?

shibing624 commented 10 months ago

1.system prompt不同; 2.格式不同

SoYuCry commented 10 months ago

我看到的 alpaca 的 prompt 是“Below is an instruction that describes a task. Write a response that appropriately completes the request. ###”,Vicuna 只有 from human 和 GPT,没有看到 prompt

shibing624 commented 10 months ago

vicuna也有system prompt.