search

microsoft / LMOps

General technology for enabling AI capabilities w/ LLMs and MLLMs
https://aka.ms/GeneralAI
MIT License
3.6k stars 274 forks source link

use_bf16_for_qwen #146

Closed SleepEarlyLiveLong closed 9 months ago

SleepEarlyLiveLong commented 9 months ago

According to the suggestion from https://github.com/microsoft/LMOps/issues/130#issuecomment-1868189639, set the data type for qwen to bfloat16 during downstream fine-tuning.