Closed boss-chanon closed 1 year ago
Add learning rate scheduler by base from https://github.com/Lightning-AI/lit-llama/blob/c3c43b6a911f3d6be5b5b3a16e844202647060e5/pretrain/redpajama.py#L299
Close #
Why this PR
Add learning rate scheduler by base from https://github.com/Lightning-AI/lit-llama/blob/c3c43b6a911f3d6be5b5b3a16e844202647060e5/pretrain/redpajama.py#L299
Changes
Related Issues
Close #
Checklist