search

aws-neuron / aws-neuron-samples

Example code for AWS Neuron SDK developers building inference and training applications
Other
101 stars 32 forks source link

Training gpt-neox 20B using neuronx-distributed #26

Closed XinweiFu closed 10 months ago

XinweiFu commented 10 months ago

This PR provides the scripts of training gpt-neox 20B using neuronx-distributed.

XinweiFu commented 10 months ago

@aws-rhsoln I got your point. I will do a revision by creating a model file which only contains the changes from the original model.

XinweiFu commented 10 months ago

Moved the PR to here: https://github.com/aws-neuron/aws-neuron-samples-staging/pull/131