axolotl-ai-cloud / axolotl

Go ahead and axolotl questions

https://axolotl-ai-cloud.github.io/axolotl/

Apache License 2.0

7.48k stars 808 forks source link

bump transformers and set roundup_power2_divisions for more VRAM improvements, low bit ao optimizers #1769

Closed winglian closed 1 month ago

winglian commented 1 month ago

Description

Motivation and Context

How has this been tested?

Screenshots (if appropriate)

Types of changes

Social Handles (Optional)