Llama2

[philschmid]

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[nth-attempt]

Thanks @philschmid for this PR. @xyang16 will this image be available in ECR as well? thanks!

[xyang16]

@nth-attempt Yes it will be available in ECR once it's released.

[monuminu]

waiting for it soon :)

[ashivadi]

Need to update the dockerfile to have proper path to the files in the text-generation-inference repo. Eg.

COPY --chmod=775 ./text-generation-inference/sagemaker-entrypoint.sh entrypoint.sh

instead of this currently here

COPY --chmod=775 sagemaker-entrypoint.sh entrypoint.sh

assuming dir structure looks like this:

$tree 
.
├── llm-hosting-container
└── text-generation-inference

[xyang16]

@ashivadi

The directory structure is like this:

$tree 
.
├── text-generation-inference
└── ── llm-hosting-container

Working directory is text-generation-inference

[ashivadi]

@ashivadi

The directory structure is like this:

$tree 
.
├── text-generation-inference
└── ── llm-hosting-container

Working directory is text-generation-inference

Thanks, updated here: https://github.com/ashivadi/llama2-with-tgi-sagemaker

[nth-attempt]

Is the image available in ECR yet?

[xyang16]

@nth-attempt The target release date is by Monday next week.

[nth-attempt]

Thank you!! https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-text-generation-inference-containers