search

a-r-r-o-w / cogvideox-factory

Memory optimized finetuning scripts for CogVideoX using TorchAO and DeepSpeed
Apache License 2.0
177 stars 16 forks source link

DeepSpeed fixes #7

Closed a-r-r-o-w closed 4 days ago

a-r-r-o-w commented 4 days ago

Based on this comment

a-r-r-o-w commented 4 days ago

The scripts already supported DeepSpeed from the beginning but there was a bug in the lr scheduler part, which has now been fixed I believe. For enabling DeepSpeed, we just need to configure with accelerate correctly (I'll push configs of those in a follow-up PR)