issues
search
YunchaoYang
/
Blogs
blogs and notes, https://yunchaoyang.github.io/blogs/
0
stars
0
forks
source link
Megatron-LM, how tensor and pipeline works
#64
Open
YunchaoYang
opened
4 months ago
YunchaoYang
commented
4 months ago
Megatron-LM/Megatron-core
Tensor-RT
FasterTransformer
YunchaoYang
commented
1 month ago
References
[源码解析] 模型并行分布式训练Megatron (2) --- 整体架构
[源码解析] 模型并行分布式训练Megatron (1) --- 论文 & 基础
[源码解析] 模型并行分布式训练 Megatron (3) ---模型并行实现
[源码解析] 模型并行分布式训练 Megatron (4) --- 如何设置各种并行
https://www.mltalks.com/posts/3016692145/
https://nn.labml.ai/optimizers/adam.html
Megatron-LM/Megatron-core
Tensor-RT
FasterTransformer