zhuohan123 terapipe issues

zhuohan123 / terapipe

65 stars 5 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Question about DP with new request

#55 lhcezx opened 4 months ago
0
Is Terapipe implemented on 1F1B schedule?

#54 robotsp opened 7 months ago
1
some question about attention module

#53 oujieww opened 2 years ago
1
Dynamic Programming Update Request

#52 ConnollyLeon opened 3 years ago
0
A question about the 6th Equation in the paper

#51 ConnollyLeon closed 3 years ago
2
Verify new implementation

#50 zhuohan123 closed 3 years ago
0
Calculate model flops and memory

#49 suquark closed 3 years ago
0
Separate the transformer model and terapipe parallelism logic

#48 zhuohan123 closed 3 years ago
0
Separate model and parallelism

#47 zhuohan123 closed 3 years ago
0
[WIP] Cleanup code for opensource

#46 zhuohan123 closed 3 years ago
0
rename pipemegatron -> terapipe

#45 zhuohan123 closed 3 years ago
0
Optimal slices for longer sequence length

#44 suquark closed 3 years ago
0
quick fix for send via broadcast

#43 zhuohan123 closed 3 years ago
0
Evaluate DP results

#42 suquark closed 3 years ago
0
Add a new dp algorithm includes batch dimension

#41 zhuohan123 closed 3 years ago
0
send/recv via broadcast

#40 zhuohan123 closed 3 years ago
1
Gradient checkpointing

#39 sguo35 closed 3 years ago
1
Fix memory model

#38 suquark closed 3 years ago
0
Benchmark script

#37 sguo35 closed 3 years ago
0
Enable inplace model initialization

#36 suquark closed 3 years ago
0
memory model

#35 suquark closed 3 years ago
0
fix backward timing

#34 zhuohan123 opened 3 years ago
0
DP latency model

#33 zhuohan123 closed 3 years ago
0
[ARCHIEVED] generate timeline

#32 suquark opened 3 years ago
0
Fix latency_model.py

#31 suquark closed 3 years ago
0
Latency Model Measurement

#30 suquark closed 3 years ago
0
Verify new concat

#29 zhuohan123 closed 3 years ago
0
Remove concatenation in attention

#28 zhuohan123 closed 3 years ago
0
Make the code shareable for both timing and benchmarking

#27 suquark closed 3 years ago
1
Simplify code

#26 suquark closed 3 years ago
2
Adjust model weights

#25 suquark closed 3 years ago
1
[DO NOT MERGE] Separate compute and comm

#24 suquark closed 3 years ago
0
Add Gpipe into pipemegatron

#23 zhuohan123 closed 3 years ago
0
Enable data parallel

#22 suquark closed 3 years ago
0
Adam & some optimizations

#21 zhuohan123 closed 3 years ago
0
move old code to archive and modify readme

#20 zhuohan123 closed 3 years ago
0
Latency decomposition

#19 suquark closed 3 years ago
0
In-place operations

#18 sguo35 closed 3 years ago
1
Implement embedding+softmax on a separate GPU

#17 sguo35 opened 3 years ago
1
Optionally init nccl communicator with MPI

#16 suquark closed 4 years ago
2
Run pipemegatron with MPI

#15 suquark closed 4 years ago
3
us-east-1 setup config

#14 suquark closed 4 years ago
0
Speedup initialization

#13 zhuohan123 closed 4 years ago
0
FP16 Mixed Precision

#12 sguo35 closed 4 years ago
2
Combine pipeline parallelism with megatron-lm

#11 zhuohan123 closed 4 years ago
0
Hot fix for pytorch 1.7.0

#10 zhuohan123 closed 4 years ago
2
Fix python interpreter path

#9 suquark closed 4 years ago
0
Refactor the codebase

#8 zhuohan123 closed 4 years ago
0
Grid search runtime for different sequence length

#7 zhuohan123 closed 4 years ago
0
Nccl fix

#6 suquark closed 4 years ago
0