search

hpcaitech / ColossalAI

Making large AI models cheaper, faster and more accessible
https://www.colossalai.org
Apache License 2.0
38.76k stars 4.34k forks source link

支持qwen吗 #6027

Closed Storm0921 closed 1 day ago

Issues-translate-bot commented 2 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Title: Support qwenuo

Storm0921 commented 2 months ago

image image https://github.com/hpcaitech/ColossalAI/blob/main/applications/ColossalChat/examples/README.md

1.如果支持qwen的话应该怎么使用呢？使用ColossalChat去sft，rm，ppo？好像没看到支持pt？ 2.Colossal-LLaMA这块是仅支持llama系列的pt和sft嘛？qwen这种和llama结构基本一致的不能套用进来？ 3.coati里看上去很多脚本，能拿来做训练吗？是干啥用的

wangbluo commented 2 months ago

Hi, Colossal-LLama is not for qwen model, as they have different prompts. You can use ColossalChat to do sft，rm，ppo but pt. If your gpu resources is limit, we recommend you to use lora strategies. Coati is for sft, and rlhf.