search

aws-neuron / aws-neuron-sdk

Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and integrated with your favorite AWS services
https://aws.amazon.com/machine-learning/neuron/
Other
454 stars 153 forks source link

[torch-neuron] GPT-2 model inference support on Inf1 #211

Open AWSGH opened 3 years ago

jerwelborn commented 3 years ago

Is there an update on this issue? Thanks!

AWSGH commented 3 years ago

Thanks for your interest. While we don't have an update to share at this moment specifically for GPT2, we are making a lot of progress on other transformer based language models like BERT. Release 1.13 has improved usability and performance significantly, I hope these are models you can get started with. If you'd like to share more details on your GPT2 use-case, please email us at: aws-neuron-support@amazon.com

kurumuz commented 3 years ago

Any progress on this?

AWSGH commented 3 years ago

Any progress on this?

Hi Eren, we do not have an update to share at the moment. We would love to learn more about your specific use-case and see if we can advise on next steps. Happy to start a discussion at: aws-neuron-support@amazon.com if you'd like.

loretoparisi commented 2 years ago

@AWSGH any news related to the GPT-2 integration?

AWSGH commented 2 years ago

Hi Loreto: thanks for your interest. While GPT style models remain on our roadmap, I do not have a detailed update as to when it will be available. To help us make an informed decision, it will be great if you can contact us with more specifics around your use-case, at aws-neuron-support@amazon.com.

sonsai123 commented 1 year ago

Hello, I'd like to run gpt-j model on AWS Inf instances. Could anybody please reply if there any update on the support for gpt style models?

eusip commented 1 year ago

Hello, I'd like to run gpt-j model on AWS Inf instances. Could anybody please reply if there any update on the support for gpt style models?

I am also interested in running GPT2/GPT-J on AWS Inf instances.

GeneralMayo commented 1 year ago

gpt2-medium is still failing during the torch_neuron.trace call for me - would be great if this was supported