awslabs / llm-hosting-container

Large Language Model Hosting Container

Apache License 2.0

74 stars 26 forks source link

Add Neuronx TGI 0.0.22 #71

Closed philschmid closed 3 months ago

philschmid commented 4 months ago

What this PR do?

This PR adds a new TGI neuron version to properly support Llama 3 instruct on inferentia.

amzn-choeric commented 4 months ago

AWS CodeBuild CI Report

CodeBuild project: tgi-pr-inf2
Commit ID: 131db0037285f6b97584cb6972bc1599c51e4ce3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

amzn-choeric commented 4 months ago

AWS CodeBuild CI Report

CodeBuild project: tgi-pr-inf2
Commit ID: a8f2be6f5fd7d064bc8c274fa95d4a01537332b4
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

amzn-choeric commented 4 months ago

AWS CodeBuild CI Report

CodeBuild project: tgi-pr-inf2
Commit ID: 83ae4837f981e703159f10975738796043d111a3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository