search

NVIDIA / nim-deploy

A collection of YAML files, Helm Charts, Operator code, and guides to act as an example reference implementation for NVIDIA NIM deployment.
https://build.nvidia.com/
Apache License 2.0
141 stars 64 forks source link

How to select a specific profile? #17

Closed bronzafa closed 4 months ago

bronzafa commented 5 months ago

When we deploy the Helm it automatically select the model profile based on the best config for the model/number of GPUs, on docker we can use the -e NIM_MODEL_PROFILE= to select a different one, but I couldn't find where to set this on the Helm chart.

supertetelman commented 4 months ago

You can still set that same environment variable using the env field of the values.yaml.

bronzafa commented 4 months ago

looks like the correct way to use it is: env: - name: NIM_MODEL_PROFILE value: "<profile_id>"