Qwen1.5 is the beta version of Qwen2, a transformer-based decoder-only language model
pretrained on a large amount of data. In comparison with the previous released Qwen, the
improvements include:
6 model sizes, including 0.5B, 1.8B, 4B, 7B, 14B, and 72B; Significant performance improvement
in Chat models; Multilingual support of both base and chat models; Stable support of 32K
context length for models of all sizes
Types of changes
[x] New feature (non-breaking change which adds functionality)
Checklist:
[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
Description
This PR introduce the QWEN family of LLMs
Qwen: comprehensive language model series
Qwen1.5 is the beta version of Qwen2, a transformer-based decoder-only language model pretrained on a large amount of data. In comparison with the previous released Qwen, the improvements include:
6 model sizes, including 0.5B, 1.8B, 4B, 7B, 14B, and 72B; Significant performance improvement in Chat models; Multilingual support of both base and chat models; Stable support of 32K context length for models of all sizes
Types of changes
Checklist: