search

awslabs / llm-hosting-container

Large Language Model Hosting Container
Apache License 2.0
75 stars 32 forks source link

feature: optimum-neuronx 0.0.16 #47

Closed jinyoung-lim closed 9 months ago

jinyoung-lim commented 9 months ago

Issue #, if available: https://sim.amazon.com/issues/SMJS-155

Description of changes:

  1. Add dockerfile for optimum-neuronx 0.0.16.
  2. Update integ test. @amzn-choeric

Tested by running in alpha account tgi-pipeline: https://tiny.amazon.com/14icwneaq/IsenLink. The pipeline was undated to include optimum build with: https://code.amazon.com/reviews/CR-111439914/revisions/1#/details.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.