search

Chris-hughes10 / pytorch-accelerated

A lightweight library designed to accelerate the process of training PyTorch models by providing a minimal, but extensible training loop which is flexible enough to handle the majority of use cases, and capable of utilizing different hardware options with no code changes required. Docs: https://pytorch-accelerated.readthedocs.io/en/latest/
Apache License 2.0
176 stars 21 forks source link

Do you know about Lightning Lite ? #15

Closed tchaton closed 2 years ago

tchaton commented 2 years ago

Hey @Chris-hughes10,

Awesome work there !

Did you know about Lightning Lite in PyTorch Lightning ? Here are the docs : https://pytorch-lightning.readthedocs.io/en/latest/starter/lightning_lite.html

lightning_lite

Chris-hughes10 commented 2 years ago

Hi @tchaton,

Thanks for the kind words!

This library was based on an internal toolkit that I've been using for a while; before Lightning lite came out!

From what I can see, Lightning Lite looks more similar to Hugging Face accelerate than PyTorch-accelerated, as it still requires you to write the core training loop; whereas PyTorch-accelerated provides this for you. In terms of features, I think that PyTorch-accelerate sits closer to PyTorch-lightning, but is much simpler, and decouples the training loop from the model and tends to work out of the box in straightforward cases. Of course, the flip side to this is that the feature set is not as broad as Lightning, but this is by design.

Overall though, I like what Lightning Lite is doing, and I support any tool which helps people focus more on the ML problem and less on infrastructure concerns such as distributed training.

Thanks, Chris