issues
search
EleutherAI
/
megatron-3d
Other
4
stars
4
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
merge from main
#28
sdtblck
closed
3 years ago
0
Persistent volume and user account
#27
joshlk
closed
3 years ago
3
Add wandb and deeperspeed
#26
joshlk
closed
3 years ago
0
Use pytorch container
#25
joshlk
closed
3 years ago
1
make index files on all local_rank_0s
#24
sdtblck
closed
3 years ago
0
broken pictures in readme
#23
thepok
closed
3 years ago
1
add small -> 2-7B train scripts
#22
sdtblck
closed
3 years ago
0
Create main.py
#21
StellaAthena
closed
3 years ago
0
Update from main
#20
StellaAthena
closed
3 years ago
0
add sparse attn
#19
sdtblck
closed
3 years ago
0
Figure out why profiler breaks after running
#18
StellaAthena
opened
3 years ago
1
Stella patches
#17
StellaAthena
closed
3 years ago
0
add flops profiler
#16
sdtblck
closed
3 years ago
0
remove bias from geglu
#15
sdtblck
closed
3 years ago
0
Add automatic detection of num_nodes
#14
StellaAthena
opened
3 years ago
0
add in our data pipeline (simplify downloading + preprocessing).
#13
StellaAthena
closed
3 years ago
0
clean up the stella_setup.sh script and save data files to marked folders instead of just the main repo directory.
#12
StellaAthena
opened
3 years ago
0
Figure out how to get data files on every node
#11
StellaAthena
closed
3 years ago
0
add in sparse attention
#10
StellaAthena
closed
3 years ago
0
add T5 positional encoding
#9
StellaAthena
opened
3 years ago
5
Add data downloaders / corpora
#8
sdtblck
closed
3 years ago
0
Add Sinusoidal Positional Embedding
#7
sdtblck
closed
3 years ago
0
add in the option not to weight-tie outputs
#6
sdtblck
closed
3 years ago
0
Add option to turn off weight tying
#5
sdtblck
closed
3 years ago
0
add in GEGLU
#4
sdtblck
closed
3 years ago
0
Add geglu
#3
sdtblck
closed
3 years ago
0
get rid of BERT stuff
#2
sdtblck
closed
3 years ago
0
Simplify, add ssh kube scripts, remove BERT stuff
#1
sdtblck
closed
3 years ago
0