mosaicml / llm-foundry

LLM training code for Databricks foundation models
https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
Apache License 2.0
3.83k stars 502 forks source link

Add `all` transforms to train script #1300

Closed dakinggg closed 1 week ago

dakinggg commented 1 week ago

In https://github.com/mosaicml/llm-foundry/pull/1290, I added a unit test for the all transform, but forgot to add it to train.py

Loss curve before and after, also verified the batch size calculations are done the same.

Screenshot 2024-06-22 at 1 52 11 PM