issues
search
xrsrke
/
pipegoose
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
MIT License
76
stars
17
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Fused Optimizer
#13
xrsrke
opened
10 months ago
8
Deparallelize tensor parallelism
#11
xrsrke
closed
9 months ago
4
Kernel Fusion using torch.jit
#10
xrsrke
opened
10 months ago
3
Dataloader and Sampler for 3D Parallelism
#9
xrsrke
closed
9 months ago
0
Port CUDA Kernels
#8
xrsrke
opened
10 months ago
6
Implement new pipeline parallelism technique
#7
xrsrke
opened
10 months ago
0
Model partitioning for pipeline parallelism
#6
xrsrke
opened
10 months ago
1
Bucket small tensors and collective operations into larger ones
#5
xrsrke
opened
10 months ago
0
Gradient Checkpointing
#4
xrsrke
opened
10 months ago
1
add test
#3
xrsrke
closed
1 year ago
0
Fsdp
#2
xrsrke
closed
1 year ago
0
WIP
#1
xrsrke
closed
1 year ago
0
Previous