issues
search
bigscience-workshop
/
Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Other
1.3k
stars
211
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Distill BLOOM - tentative 2
#354
younesbelkada
opened
1 year ago
0
Enable rocm-support
#353
luukkonenr
opened
1 year ago
0
Distill megatron - test Draft WIP
#352
younesbelkada
closed
1 year ago
0
Distill megatron - WIP draft code
#351
younesbelkada
closed
1 year ago
0
Load Bloom Optimizer State (i.e. Bloom 1B1)
#350
philippmtk
opened
1 year ago
2
Encoding checkpoint reshaping guide
#349
tjruwase
opened
1 year ago
1
Slower inference results for BLOOM fp16 on identical hardware
#348
sarthaklangde
opened
1 year ago
5
grad norm increase strangely
#347
misska1
opened
1 year ago
12
How to inference GPT2 with DeepSpeed?
#346
cdj0311
closed
1 year ago
1
[bloom inference scripts] improvements
#345
stas00
closed
1 year ago
0
[Bloom inference] further improvements
#344
stas00
closed
1 year ago
1
About reshape deepspeed checkpoint
#343
henan991201
opened
1 year ago
20
Installing Apex on Windows
#342
gordicaleksa
opened
1 year ago
1
pretrain_gpt_distributed.sh ERROR!
#341
cdj0311
closed
1 year ago
0
[ds-inference bloom] tweaks
#340
stas00
closed
1 year ago
4
Followup PR for adding generation-server
#339
mayank31398
closed
1 year ago
12
About convert deepspeed to deepspeed checkpoint
#338
henan991201
opened
1 year ago
4
Finetuning BLOOM
#337
AnaRhisT94
opened
2 years ago
5
Add multiple evaluation compat
#336
Muennighoff
opened
2 years ago
0
Changing a single example affects forward pass for other examples in a batch
#335
mayank31398
closed
2 years ago
4
Can we also train BLOOM model using tensor using tensor-Parallelism and efficient fused CUDA kernels
#334
CloudedLeopard17
opened
2 years ago
4
About convert DS checkpoint to Transformers
#333
misska1
closed
1 year ago
2
disable CI
#332
stas00
closed
2 years ago
1
merge main
#331
Muennighoff
closed
2 years ago
0
DeepSpeed inference support for int8 parameters on BLOOM?
#330
pai4451
closed
2 years ago
6
how to convert huggingface model to megatron-deepspeed?
#329
yayaQAQ
closed
1 year ago
8
Add generation server scripts using HF accelerate and DS-inference
#328
mayank31398
closed
1 year ago
46
[checkpoints] replace bf16 with fp32 checkpoint weights
#327
stas00
opened
2 years ago
3
Add option to normalize loss per target
#326
Muennighoff
closed
1 year ago
0
Add generation server scripts
#325
mayank31398
closed
2 years ago
1
Errors in generation (Bloom) when changing options sampling/use_cache
#324
thies1006
opened
2 years ago
29
Question about downloading checkpoints of 6.3B，2.5B，1.3B
#323
misska1
opened
2 years ago
3
add args_deepspeed_gpt.sh
#322
xyn1201
closed
2 years ago
0
Generation server using HF accelerate and DS inference
#321
mayank31398
closed
2 years ago
19
"Mask is silently ignored due to the use of a custom kernel" with pretrain_gpt_single_node.sh
#320
tianjianjiang
opened
2 years ago
4
where can I download the 176B checkpoint in deepspeed format?
#319
xuyifan-0731
opened
2 years ago
17
Multi-node inference with Bloom: Unhandled CUDA error in ProcessGroupNCCL.cpp (called from all_reduce in torch)
#318
asaparov
opened
2 years ago
31
Question: How to run multi-node inference?
#317
asaparov
closed
2 years ago
1
add OnDevice and remove zero-inference
#316
jeffra
closed
2 years ago
0
Question about downloading checkpoints
#315
mayank31398
closed
2 years ago
2
How to run generation?
#314
mayank31398
closed
2 years ago
1
Prefix LM Eval
#313
Muennighoff
opened
2 years ago
4
DeepSpeed checkpoint performance improvements
#312
adammoody
opened
2 years ago
0
Add Bitfit
#311
Muennighoff
opened
2 years ago
0
Tool for CKPT averaging
#310
Muennighoff
opened
2 years ago
0
Enable loading ckpt for t0 finetuning
#309
Muennighoff
opened
2 years ago
0
BLOOM Inference via DeepSpeed-Inference, Accelerate and DeepSpeed-ZeRO
#308
stas00
closed
2 years ago
46
Add bias a weight we need to sync as well across TP
#307
thomasw21
closed
2 years ago
0
Fix causal attention mask
#306
thomasw21
closed
2 years ago
0
Broke causal masking for gpt_model with used with deepspeed without fused kernels
#305
thomasw21
closed
2 years ago
0
Previous
Next