Adds support for float values for val_check_interval for SFT. It also adds support for floats/ints for limit_train_batches to SFT and DPO, as per the usage in PTL
This was requested by @Kipok
Changelog
Please update the CHANGELOG.md under next version with high level changes in this PR.
Usage
You can potentially add a usage example below
val_check_interval = 0.25 # means you will run validation 4 times per epoch
val_check_interval = 100 # means you will run validation every 100 steps of training
limit_train_batches = 0.5 # you will only use 50% of your training data per epoch
limit_train_batches = 100 # you will only consume 100 steps of your train dataloader per epoch
All possibilities can be used for SFT, DPO, and SPIN
What does this PR do ?
Adds support for float values for
val_check_interval
for SFT. It also adds support for floats/ints forlimit_train_batches
to SFT and DPO, as per the usage in PTLThis was requested by @Kipok
Changelog
Usage
All possibilities can be used for SFT, DPO, and SPIN
Before your PR is "Ready for review"
Pre checks:
Checklist when contributing a new algorithm
max_steps=-1
andvalidation
?Additional Information