Closed jaywonchung closed 3 months ago
HuggingFace's SFTTrainer (https://huggingface.co/docs/trl/main/en/sft_trainer) inherits from Trainer and supports callbacks, so in theory it should just work with HFGlobalPowerLimitOptimizer (example in https://github.com/ml-energy/zeus/tree/master/examples/huggingface). We should quickly check fine-tuning with SFTTrainer with HFGlobalPowerLimitOptimizer and add an example.
SFTTrainer
Trainer
callbacks
HFGlobalPowerLimitOptimizer
HuggingFace's
SFTTrainer(https://huggingface.co/docs/trl/main/en/sft_trainer) inherits fromTrainerand supportscallbacks, so in theory it should just work withHFGlobalPowerLimitOptimizer(example in https://github.com/ml-energy/zeus/tree/master/examples/huggingface). We should quickly check fine-tuning withSFTTrainerwithHFGlobalPowerLimitOptimizerand add an example.