issues
search
iwiwi
/
epochraft-hf-fsdp
Example of using Epochraft to train HuggingFace transformers models with PyTorch FSDP
MIT License
12
stars
5
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Support Finetune with peft(First, only LoRA)
#17
shimatar0
opened
7 months ago
0
Add configs
#16
iwiwi
closed
10 months ago
0
Fix OmegaConf type errors with Python 3.8
#15
iwiwi
closed
10 months ago
0
Add 'labels' field in each sample
#14
iwiwi
closed
10 months ago
0
Support BetterrTransformer
#13
iwiwi
closed
10 months ago
0
Support BetterTransformers
#12
iwiwi
closed
10 months ago
2
Support `use_flash_attention_2` under `fsdp_low_cpu_init` option
#11
iwiwi
opened
10 months ago
0
Initialization with low CPU memory usage
#10
iwiwi
closed
10 months ago
4
Add model_kwargs config
#9
iwiwi
closed
10 months ago
0
Support Llama-2 70B
#8
iwiwi
opened
10 months ago
1
Fix memory consumption issues
#7
iwiwi
closed
10 months ago
0
Add use_flash_attention_2=True
#6
iwiwi
closed
10 months ago
0
Swap the order of validation and ckpting as a workaround for device errors
#5
iwiwi
closed
11 months ago
0
More precise FLOPs computation
#4
iwiwi
opened
11 months ago
0
Add CPU offloading option
#3
iwiwi
closed
10 months ago
1
Refactor to simplify `train.py`
#2
iwiwi
closed
11 months ago
0
Example code for supervised finetuning using epochraft + HF FSDP
#1
fujiki-1emon
opened
11 months ago
1