search

mosaicml / llm-foundry

LLM training code for Databricks foundation models
https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
Apache License 2.0
3.99k stars 525 forks source link

Bump transformers version to 4.43.1 #1388

Closed dakinggg closed 2 months ago

dakinggg commented 2 months ago

Loss goes down on llama 3.1

Screenshot 2024-07-23 at 11 06 47 AM

Simple test llama 2: transformers-upgrade-1-wVJYPy Llama 3.1 test: llama-31-3-Ha2mkM