search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.2k stars 21.35k forks source link

How to use environment variables in entry script during AML deployment process #82630

Closed sunnyhay closed 2 years ago

sunnyhay commented 2 years ago

Hello, Our team now encounters a problem when trying to use environment variables in entry script. For example, in entry script we’d like to get AppInsight key this way APPINSIGHTKEY = os.environ.get('APPINSIGHTKEY')

And the place to inject the environment variable is Azure DevOps pipeline. Since we have a variety of environments (Dev/PPE/Prod), we’d like to use different values for a specific environment variable. The tricky part is that we build AKS cluster first and then attach it with existing AML workspace. Finally we deploy the model and entry script on the AKS cluster. Above operations are performed through AML CLI. But in the last step of model deployment, an error will be detected in the entry script if we try to use environment variables in the entry script like above AppInsight example. I know AML environment is not been associated with the provisioned AKS cluster and so the environment variable does not work with our entry script. I use recommended AML CLI to update the existing deployment with environment variables and it works. But that’s the update service step, not the first-time deployment. Do you know any best practice to inject the environment variables into entry script when the first-time deployment is made in this context? Environment files like .env is not good since we do not want to check in secrets into our code. Also Azure DevOps pipelines allow pipeline variables and we’d like to see if/how those can be used with entry script. Thank you!

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ram-msft commented 2 years ago

@sunnyhay Thanks for the question. We are investigating the issue and will update you shortly.

ram-msft commented 2 years ago

@sunnyhay Thanks, Can you check os.environ that returns the environment variables. I do see the Inference Config Class has a spot for environment which works with the Model.deploy method.

The SDK simplifies by abstracting the underpinning around image creation etc. And if your scenario demands securely accessing your Azure resources without having to embed credentials/secrets inside your usage script you can find details on how to do so via Azure AD Identity . https://docs.microsoft.com/en-us/azure/machine-learning/how-to-use-azure-ad-identity.

doing this manually in AKS via: • Defining an Environment Variable in the AKS configuration file • Using secrets in your AKS Pod and using secrets as environment variables • Grabbing an existing deployment and editing the configuration via kubectl get deployments -n namespace and kubectl edit deploymentname

ram-msft commented 2 years ago

We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.