Training framework integration opportunities - Githubissues

ml-energy / zeus

Deep Learning Energy Measurement and Optimization

https://ml.energy/zeus

Apache License 2.0

179 stars 24 forks source link

Training framework integration opportunities #77

Open jaywonchung opened 1 month ago

jaywonchung commented 1 month ago

PipelineFrequencyOptimizer: Large model training frameworks
- Deepspeed
- Megatron-LM
- Deepspeed-Megatron
- GPT-NeoX
- Reuse training examples in each large model training framework (e.g., Llama pre-training or fine-tuning)
GlobalPowerLimitOptimizer
- PyTorch FSDP
- Deepspeed ZeRO