search

QwenLM / Qwen

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
Apache License 2.0
13.59k stars 1.11k forks source link

14b模型能微调支持32K吗 #998

Closed Longleaves closed 7 months ago

Longleaves commented 8 months ago

起始日期 | Start Date

No response

实现PR | Implementation PR

No response

相关Issues | Reference Issues

No response

摘要 | Summary

14b模型能微调支持32K吗

基本示例 | Basic Example

无

缺陷 | Drawbacks

无

未解决问题 | Unresolved questions

No response

jklj077 commented 7 months ago

In the recently unveiled Qwen1.5 release, all Qwen1.5 models, including Qwen1.5-14B boast enhanced capability, supporting sequence lengths up to an impressive 32,000 tokens.

For comprehensive discussions about these changes, further details on the updated architecture, and any related inquiries, please visit the official Qwen1.5 GitHub repository at https://github.com/QwenLM/Qwen1.5.