search

QwenLM / Qwen2.5-Coder

Qwen2.5-Coder is the code version of Qwen2.5, the large language model series developed by Qwen team, Alibaba Cloud.
3.1k stars 210 forks source link

关于补全模型微调 #82

Closed Soulscb closed 4 months ago

Soulscb commented 4 months ago

你好，我们想微调base模型的补全能力，但是发现你们的chat模型有单独的格式，请问需要遵从你们的微调格式吗？

cyente commented 4 months ago

建议暂时不要使用微调格式

80

Soulscb commented 4 months ago

感谢您的回复，请问数据拼接过程中，bos和eos的拼接是怎么做的呢？微调FIM的格式是否是 bos + input + output + eos

cyente commented 4 months ago

https://github.com/QwenLM/CodeQwen1.5/blob/main/examples/CodeQwen1.5-base-fim.py

fim的格式参考此处example