search

aws-neuron / aws-neuron-samples

Example code for AWS Neuron SDK developers building inference and training applications
Other
101 stars 32 forks source link

Deploy meta-llama-2-13b-sampling.ipynb on inf2.24xlarge #50

Closed rajamdhi closed 8 months ago

rajamdhi commented 9 months ago

Hi,

Is it possible to deploy meta-llama-2-13b-sampling.ipynb on inf2.24xlarge machine?.

aws-taylor commented 8 months ago

Hello @rajamdhi,

Thank you for your issue. Take a look at the related issue https://github.com/aws-neuron/aws-neuron-sdk/issues/741. In short - yes - it should be possible to use an inf2.24xlarge as long as you configure your context length appropriately.

Regards, Taylor

mrnikwaws commented 8 months ago

Closing since we have not had any further questions on this topic