issues
search
tdrussell
/
qlora-pipe
A pipeline parallel training script for LLMs.
MIT License
79
stars
5
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Training crashes with Accelerate 0.33.0
#22
rationalism
opened
1 week ago
0
feature: proportional eval steps
#21
kallewoof
opened
2 weeks ago
0
LoRA+ support
#20
kallewoof
closed
2 days ago
5
multi-node training error during eval/best_loss checkpoint saving: Watchdog caught collective operation timeout: WorkNCCL(SeqNum=2315, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1808543 milliseconds before timing out.
#19
iamhappytoo
opened
1 month ago
2
Softcapping for lm head logits?
#18
kallewoof
closed
1 month ago
2
qol: write out a UTF-8 plot on screen showing eval loss history, if plotille is installed
#17
kallewoof
closed
1 month ago
3
How to correctly set bf16 for training, and seems deepspeed config additional offload/bf16 conflicts with the qlora-pipe setting?
#16
iamhappytoo
opened
2 months ago
4
bug-fix: use right varname in lm_head
#15
kallewoof
closed
2 months ago
0
qol: show ETA as a human readable time
#14
kallewoof
closed
3 months ago
0
feature: track and hold onto best eval model
#13
kallewoof
closed
2 months ago
0
feature: add --resume_from_checkpoint flag
#12
kallewoof
closed
3 months ago
0
trivial: allow commented-out resume_from_checkpoint config to mean 'false'
#11
kallewoof
closed
3 months ago
0
merge_lora: detect and use safetensors lora when present
#10
kallewoof
closed
3 months ago
0
add old-state trimming and handle saving in new module
#9
kallewoof
closed
3 months ago
2
improve steps logging including adding an ETA
#8
kallewoof
closed
3 months ago
1
fix: avoid empty batches by including incomplete batches when there are no complete ones
#7
kallewoof
closed
3 months ago
6
qol: warn user when they are about to spend a lot of time evaluating vs training
#6
kallewoof
closed
3 months ago
2
merge_lora: switch copy order and add tqdm
#5
kallewoof
closed
3 months ago
0
cleanup: fix imports
#4
kallewoof
closed
3 months ago
0
fix: deal gracefully with warmup_steps=0
#3
kallewoof
closed
3 months ago
0
DistributedBatchSamper crashes on small eval sample sizes
#2
accupham
closed
3 months ago
1
Could you provide an example of a 70B model training config?
#1
accupham
opened
5 months ago
4