search

facebookresearch / optimizers

For optimization algorithm research and development.
Other
442 stars 31 forks source link

Using Shampoo with Accelerate and FSDP #24

Closed kfirgoldberg closed 2 weeks ago

kfirgoldberg commented 1 month ago

Hi, Does the Shampoo implementation support HuggingFace's Accelerate library? Can it be used in: model, optimizer, scheduler = accelerator.prepare(model, optimizer, scheduler) ?

Thanks!

tsunghsienlee commented 1 month ago

Hi @kfirgoldberg ,

We haven't tried that with HuggingFace's Accelerate library? I wonder have you tried that, and did you encounter any issue?

kfirgoldberg commented 1 month ago

I didn't try it yet, I'll update if I get to it in the future

tsunghsienlee commented 2 weeks ago

Hi @kfirgoldberg ,

It has been one month from now since my last reply, and if you did encounter issue running HuggingFace's Accelerate library, please come back and reopen this issue.