Open tensiondriven opened 1 year ago
The finetune scripts use transformers.Trainer, so you can just utilize the feature of it. You can adjust the finetune.py:
class MyCallback(transformers.TrainerCallback):
"A callback that prints a message at the beginning of training"
def on_step_end(self, args, state, control, **kwargs):
if len(state.log_history) > 0:
if state.log_history[-1]['loss'] < 1.5:
control.should_training_stop = True
...
trainer = transformers.Trainer(
...
callbacks=[MyCallback],
)
And you can also check its document: https://huggingface.co/docs/transformers/main_classes/callback
Several people on Reddit have been mentioning ideal loss ranges for training. Some training tools have an option to "stop training when loss reaches X". I would love to use this feature with alpaca_lora_4bit, so I wouldn't have to guess at the ideal loss, save a lot of checkpoints, etc.
Would this be something feasible to implement? I may be missing something about the details which would prevent it from being feasible.
For example, I would love to be able to specify: