update inference ami version in sagemaker endpoint config to fix nvml driver issue

NVIDIA / nim-deploy

A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.

https://build.nvidia.com/

Apache License 2.0

143 stars 64 forks source link

update inference ami version in sagemaker endpoint config to fix nvml driver issue #104

Closed kshitizgupta21 closed 1 week ago

kshitizgupta21 commented 1 week ago

This PR updates the nim_llama3.ipynb jupyter notebook to use 'InferenceAmiVersion': 'al2-ami-sagemaker-inference-gpu-2' within ProductionVariants inside EndpointConfig. This will make sure that newer driver is used on g5 and p4d/p4de instances instead of default 470 one and solve pynvml driver issues. It should fix this issue https://github.com/NVIDIA/nim-deploy/issues/98

cc: @JamesMaki @abhisheksawarkar