search

QwenLM / Qwen2.5-Coder

Qwen2.5-Coder is the code version of Qwen2.5, the large language model series developed by Qwen team, Alibaba Cloud.
3.1k stars 210 forks source link

Question about training datasets #56

Closed Owen-Qin closed 6 months ago

Owen-Qin commented 6 months ago

Hi Qwen team,

Thanks for your great work! I just have two quick questions.

  1. Does the pretaining dataset contain the new released stack v2?
  2. Could you shed some light on the creation of the SFT dataset? For example, did you use synthesis data? What is the magnitude of the SFT data?

Will there be a technical paper for CodeQwen?

cyente commented 6 months ago

Thank you for your appreciation. We didn't use stack v2; instead, we built our own training set. Please wait for our technical report for more details about our training.