search

aws-neuron / aws-neuron-sdk

Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and integrated with your favorite AWS services
https://aws.amazon.com/machine-learning/neuron/
Other
462 stars 154 forks source link

[PyTorch] PyTorch Lightning support #543

Open aws-rxgupta opened 2 years ago

aws-rxgupta commented 2 years ago

[torch-neuronx] PyTorch Lightning support

dennj commented 11 months ago

What version of Lightning are you targeting? 2.x or 1.x also?

blessondavis commented 6 months ago

Is there anyone who is working on creating this support?

aws-rhsoln commented 6 months ago

We have support for PyTorch-lightning using neuronx-distributed library. Here are couple of examples to help you get started:

  1. https://awsdocs-neuron.readthedocs-hosted.com/en/latest/libraries/neuronx-distributed/tutorials/training_llama2_tp_pp_ptl.html#llama2-tp-pp-ptl-tutorial
  2. https://awsdocs-neuron.readthedocs-hosted.com/en/latest/libraries/neuronx-distributed/tutorials/training_llama2_tp_pp_ptl.html#llama2-tp-pp-ptl-tutorial