search

facebookresearch / schedule_free

Schedule-Free Optimization in PyTorch
Apache License 2.0
1.91k stars 65 forks source link

8bit optimizers #40

Closed winglian closed 3 months ago

winglian commented 3 months ago

Would it be possible to enable 8bit optimizers with kahan summation for lower vram requirements?

adefazio commented 3 months ago

We don't have any immediate plans to work on 8-bit versions. There are several projects that maintain 8-bit versions of common optimizers, they might be interested in adding Schedule-Free versions.