search

clue-ai / ChatYuan

ChatYuan: Large Language Model for Dialogue in Chinese and English
https://www.clueai.cn
Other
1.9k stars 183 forks source link

分布式训练 是 数据并行 还是 模型并行？感觉要 模型并行 才能成为大模型，是吧？ #32

Closed guotong1988 closed 1 year ago

guotong1988 commented 1 year ago

https://github.com/clue-ai/ChatYuan/tree/main/distributed-training

guotong1988 commented 1 year ago

https://github.com/NVIDIA/Megatron-LM