4xH100 Training Preparation Check List

homebrewltd / research

1 stars 0 forks source link

Closed hungphongtrn closed 2 months ago

hungphongtrn commented 2 months ago

[ ] Dependencies: [ ] torch 2.3 [ ] sudo [ ] libaio-dev [ ] transformers datasets deepspeed accelerate tensorboardX bitsandbytes trl peft flash-attn [ ] clone cutlass [ ] export cutlas path, cflags, ldflags, gpus
[ ] Data: split text and sound -> push final dataset for training
[ ] Training Code
[ ] Accelerator Config

tikikun commented 2 months ago

it's done