search

QwenLM / CodeQwen1.5

CodeQwen1.5 is the code version of Qwen, the large language model series developed by Qwen team, Alibaba Cloud.
371 stars 22 forks source link

Question about training datasets #56

Closed Owen-Qin closed 1 month ago

Owen-Qin commented 1 month ago

Hi Qwen team,

Thanks for your great work! I just have two quick questions.

  1. Does the pretaining dataset contain the new released stack v2?
  2. Could you shed some light on the creation of the SFT dataset? For example, did you use synthesis data? What is the magnitude of the SFT data?

Will there be a technical paper for CodeQwen?

cyente commented 1 month ago

Thank you for your appreciation. We didn't use stack v2; instead, we built our own training set. Please wait for our technical report for more details about our training.