mosaicml / llm-foundry

LLM training code for Databricks foundation models
https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
Apache License 2.0
3.83k stars 502 forks source link

Ignore mosaicml logger for exception if excephook is active #1301

Closed jjanezhang closed 1 week ago

jjanezhang commented 1 week ago

Ignore mosaicml logger for exception if excephook is active

Before we remove the logger all together (re: https://github.com/mosaicml/llm-foundry/pull/1292) we want to first test catching exceptions with a flag. This prevents us from logging an exception twice. I will run tests that turn the logger off in favor of overriding the excepthook to make sure the intended before still holds for finetuning.

Ran a test on dbx-staging to confirm that the flag works. We will have to update the foundry image used for finetuning to incorporate this change. image