想问下预训练长下文长度是多少？

QwenLM / Qwen2.5-Coder

Qwen2.5-Coder is the code version of Qwen2.5, the large language model series developed by Qwen team, Alibaba Cloud.

3.04k stars 202 forks source link

Closed robinsonmd closed 7 months ago

huybery commented 7 months ago

8k pretrain, 64k continue train.