search

mosaicml / composer

Supercharge Your Model Training
http://docs.mosaicml.com
Apache License 2.0
5.15k stars 418 forks source link

PyTorch Lightning integration collaboration #783

Closed williamFalcon closed 1 year ago

williamFalcon commented 2 years ago

Love the tricks introduced by mosaic!

We created an issue to enable the broader pytorch community access to these tricks if they want to enable the mosaic optimizations.

Excited to collaborate with you guys to make sure it's done correctly and future proof so that you can continue adding new features without code changes on the Lightning side.

https://github.com/PyTorchLightning/pytorch-lightning/issues/12360

hanlint commented 2 years ago

Sounds good, looking forward to collaborate! We care most about delivering efficiency methods to the community in whichever form they would like to consume.

heitorrapela commented 2 years ago

Hello, @williamFalcon and @hanlint.

Do you have any news about this feature?

I am using pytorch lightning, but I think it would be useful to add mosaic training strategies like those proposed in https://github.com/Lightning-AI/lightning/issues/12360. Is there a way to combine both trainers easily?

Thanks for the effort in developing awesome libraries.

GeorgePearse commented 2 years ago

Yeah, what's happened to this work? Seemed really exciting, then sort of disappeared again.

mvpatel2000 commented 1 year ago

Closing since we're not actively planning on integrating speedup methods with lightning. We're happy to discuss and accept / review community PRs though!