Right now include_tokens_per_second=True in Trainer only reports the tokens per second metric at the end of training. It would be very useful to have this metric reported continuously inside the training loop so we can monitor it during training.
Motivation
The current behavior is counter-intuitive, doesn't align with other convenient trainers (like torchtune), and it's undocumented, so I had to RTFC to figure out why the metric wasn't showing up.
Your contribution
I probably don't have time to contribute it myself.
Rocketknight1
commented
1 week ago
Feature request
Right now
include_tokens_per_second=TrueinTraineronly reports the tokens per second metric at the end of training. It would be very useful to have this metric reported continuously inside the training loop so we can monitor it during training.Motivation
The current behavior is counter-intuitive, doesn't align with other convenient trainers (like torchtune), and it's undocumented, so I had to RTFC to figure out why the metric wasn't showing up.
Your contribution
I probably don't have time to contribute it myself.