az ml model deploy is failing occasionally with 'Error': MlCliError({'Error': 'Error parsing --deploy-config-file.

[saroup]

Extension name (the extension in question)

Azure-cli-ml Version': '1.3.0'

Description of issue (in as much detail as possible)

In my pipeline I am deploying a model to ACI:

• stage: 'Deploy_ACI' displayName: 'Deploy to ACI' dependsOn: Trigger_AML_Pipeline condition: and(or(succeeded(), variables['MODEL_BUILD_ID']),variables['ACI_DEPLOYMENT_NAME']) jobs:
  • job: "Deploy_ACI" displayName: "Deploy to ACI" container: mlops timeoutInMinutes: 0 steps:
  • template: model-version-template.yml
  • task: ms-air-aiagility.vss-services-azureml.azureml-model-deploy-task.AMLModelDeploy@0 displayName: 'Azure ML Model Deploy' inputs: azureSubscription: $(WORKSPACE_SVC_CONNECTION) modelSourceType: manualSpec modelName: '$(MODEL_NAME)' modelVersion: $(MODEL_VERSION) inferencePath: '$(Build.SourcesDirectory)/$(SOURCES_DIR_TRAIN)/scoring/inference_config.yml' deploymentTarget: ACI deploymentName: $(ACI_DEPLOYMENT_NAME) deployConfig: '$(Build.SourcesDirectory)/$(SOURCES_DIR_TRAIN)/scoring/deployment_config_aci.yml' overwriteExistingDeployment: true

It sometimes fails (and then passes without any changes) with the following error: Command: az ml model deploy -n projectaci --model project_model.pkl:31 --ic /w/1/s/project/scoring/inference_config.yml --dc /w/1/s/project/scoring/deployment_config_aci.yml -g project-mlops -w projectml --overwrite 2020-04-21T12:16:58.6444046Z [command]/usr/local/envs/mlopspython_ci/bin/az ml model deploy -n projectaci --model project_model.pkl:31 --ic /w/1/s/project/scoring/inference_config.yml --dc /w/1/s/project/scoring/deployment_config_aci.yml -g project -w projectml --overwrite 2020-04-21T12:17:02.6614303Z ERROR: {'Azure-cli-ml Version': '1.3.0', 'Error': MlCliError({'Error': 'Error parsing --deploy-config-file. Must be valid JSON or YAML file.', 'Response Content': TypeError("deploy_configuration() got an unexpected keyword argument 'vnet_name'")})} 2020-04-21T12:17:02.6617007Z Output:
2020-04-21T12:17:02.6620245Z ##[error]Error Code: [1] 2020-04-21T12:17:02.6623403Z ##[error]Error: Failed to deploy model(s) 2020-04-21T12:17:02.6648147Z { code: 1, 2020-04-21T12:17:02.6648952Z error: undefined, 2020-04-21T12:17:02.6649605Z stdout: '', 2020-04-21T12:17:02.6651208Z stderr: 'ERROR: {\'Azure-cli-ml Version\': \'1.3.0\', \'Error\': MlCliError({\'Error\': \'Error parsing --deploy-config-file. Must be valid JSON or YAML file.\', \'Response Content\': TypeError("deploy_configuration() got an unexpected keyword argument \'vnet_name\'")})}\n' } 2020-04-21T12:17:02.6655320Z Reverting to original directory..... 2020-04-21T12:17:02.6660191Z /__w/1/s 2020-04-21T12:17:02.6891935Z ##[section]Finishing: Azure ML Model Deploy

deployment_config_aci.yml : computeType: ACI containerResourceRequirements: cpu: 1 memoryInGB: 4

Any hints on how I can debug this further or the root cause of the problem?

[yonzhan]

ml

[amitsethitx]

I am seeing similar error . Is there any solution ?

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

[azureml-github]

@AML on Azure DevOps Supportmailto:aml_ado_support@microsoft.com / @MLOps Coremailto:mlopscore@microsoft.com / @Mrudula Narkhedemailto:mnark@microsoft.com

Anyone seen this before? I suspect it relates to ACI VNET support.

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: msftbot[bot] notifications@github.com Sent: Monday, September 14, 2020 4:55:10 PM To: Azure/azure-cli-extensions azure-cli-extensions@noreply.github.com Cc: azureml-github azureml-github@microsoft.com; Mention mention@noreply.github.com Subject: Re: [Azure/azure-cli-extensions] az ml model deploy is failing occasionally with 'Error': MlCliError({'Error': 'Error parsing --deploy-config-file. (#1573)

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-githubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fazureml-github&data=02%7C01%7CJordane%40microsoft.com%7Cb276ce01125946c2384f08d859099dbd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637357245596607327&sdata=Ew6RzgulLVoxya2WR%2F90DJbNzts7W%2BmufM1mw5P7%2F0E%3D&reserved=0.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-cli-extensions%2Fissues%2F1573%23issuecomment-692378752&data=02%7C01%7CJordane%40microsoft.com%7Cb276ce01125946c2384f08d859099dbd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637357245596607327&sdata=rnh1TRygcDyU8JI7nhndTVgho9sTw5iomBFOP3cwtms%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOVYYXSXOHUWA6OAJSNWD2DSF2UN5ANCNFSM4MNIS4BQ&data=02%7C01%7CJordane%40microsoft.com%7Cb276ce01125946c2384f08d859099dbd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637357245596607327&sdata=wnPBIeNBJjg3ITmRPknPzwTPehK3Glhca8mNY07f2Wo%3D&reserved=0.

[Bozhong68]

@saroup @yonzhan did you attempt to deploy model to ACI with VNET support? Note ACI deployment doesn't support VNET yet. Please try AKS deployment if you want VNET support.

[yonzhan]

machine learning service team should look into this.

[lostmygithubaccount]

@saroup are you still facing this issue?

[v-strudm-msft]

Since there hasn't been a response to our last comment, we are closing this issue. Should there still be a question, please open a new issue. Thank you.