search

awslabs / llm-hosting-container

Large Language Model Hosting Container
Apache License 2.0
75 stars 32 forks source link

Add Optimum Neuronx TGI image 0.0.25 #100

Closed dacorvo closed 2 weeks ago

dacorvo commented 2 weeks ago

This new image is based on AWS Neuron SDK 2.20.

It adds support for LLama 3.1-3.2 models.