issues
search
SparkJiao
/
llama-pipeline-parallel
A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to copy code and launch discussions about the problems you have encoured.
44
stars
2
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Implementing Pipeline Parallelism with LLaMA Models and Utilizing deepspeed for Execution?
#7
ShuaipengWu
opened
4 months ago
3
How Can I use your code to load LLama2?
#6
fmh1art
closed
5 months ago
5
RuntimeError: element 1 of tensors does not require grad and does not have a grad_fn
#5
OAfzal
closed
5 months ago
4
pipeline with zero1, does not optimize gpu meomery
#4
xinpeng-zhang
closed
11 months ago
3
求助求助
#3
newtonysls
closed
1 year ago
11
hey bro! your project help me a lot! You are my hero!!!
#2
laiqinghan
closed
1 year ago
1