想问下预训练长下文长度是多少？

QwenLM / CodeQwen1.5

CodeQwen1.5 is the code version of Qwen, the large language model series developed by Qwen team, Alibaba Cloud.

371 stars 22 forks source link

Closed robinsonmd closed 2 months ago

huybery commented 2 months ago

8k pretrain, 64k continue train.