adefazio
commented
1 year ago They don't have native support. I've used them within fairseq (which provides float16 support by wrapping the optimizer) in some of my experiments.
Closed TKassis closed 1 year ago
adefazio
commented
1 year ago They don't have native support. I've used them within fairseq (which provides float16 support by wrapping the optimizer) in some of my experiments.
Just to confirm, these optimizes don't support 16 bit precision training yet, correct?