search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.49k stars 2.61k forks source link

AzureResourceManagerTemplateDeployment fails on multiline strings #14645

Closed SDedik closed 3 years ago

SDedik commented 3 years ago

Required Information

Type: Bug

Enter Task Name: AzureResourceManagerTemplateDeployment@3

Environment

Issue Description

Consider following very minimal ARM template poc.json:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-08-01/subscriptionDeploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "parameter1": {
            "type": "string",
            "defaultValue": "abc",
            "metadata": {
                "description": "very long
                and multiline
                description"
            }
        }
    },
    "functions": [],
    "variables": {},
    "resources": [],
    "outputs": {}
}

and Azure Pipeline to run it:

trigger: none

pool: 
  vmImage: 'ubuntu-latest'

steps:
- task: AzureResourceManagerTemplateDeployment@3
  inputs:
    deploymentScope: Subscription
    azureResourceManagerConnection: 'my service connection 1'
    subscriptionId: 40f2d467-c27b-4ded-a7bc-82dc0b95cbb2
    templateLocation: Linked artifact
    csmFile: poc.json
    location: eastus

Pipeline fails with the following error message **##[error]Check out the troubleshooting guide to see if your issue is addressed: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting

[error]Error: Ensure the Template file ( '/home/vsts/work/1/s/poc.json' ) is valid. Task failed while parsing with following error: Unexpected token

in JSON at position 321**

Running the template manually from PowerShell like this works without any problems New-AzSubscriptionDeployment -Location 'eastus' -TemplateFile .\poc.json

As soon as the parameter description in the ARM template is changed from multiline string to single line like so

        "parameter1": {
            "type": "string",
            "defaultValue": "abc",
            "metadata": {
                "description": "single line description"
            }
        }

the pipeline task works as expected

tejasd1990 commented 3 years ago

Multiline strings like the one in your example are not valid in JSON, a JSON parser/editor will highlight the error. Multilple lines need to be separated by \n. You will not face issues with a valid JSON template file. The task does JSON parsing to determine that the template is a valid JSON. The error you see is thrown during validations done by the task before the deployment is don, not during deployment. It appears that az cli and az powershell might not make this check, so the deployment succeeds.

This is by design and not a bug in the task, so will close it.

SDedik commented 3 years ago

Please see the https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/template-syntax#multi-line-strings

Multiline strings are supported in ARM template json. Plus, applying the template via PowerShell cmdlet works without any problem, proving once again that json is valid. Also, see very related issue reported here: https://github.com/microsoft/azure-pipelines-tasks/issues/14268 which is basically the same issue, but reported for a different Task.

SDedik commented 3 years ago

The error you see is thrown during validations done by the task before the deployment is don, not during deployment.

Exactly. During, or to be more precise, right before the deployment the json is validated once again by the Azure Deployment engine (Resource Manager?). So why validate it twice: by the task and by Azure itself? Isn't Azure knows better what is valid and what is not?

bishal-pdMSFT commented 3 years ago

@SDedik could you please attach debug logs for the failure

SDedik commented 3 years ago

Sure thing, please see full logs attached below

Debug Logs ``` ##[debug]Evaluating condition for step: 'AzureResourceManagerTemplateDeployment' ##[debug]Evaluating: SucceededNode() ##[debug]Evaluating SucceededNode: ##[debug]=> True ##[debug]Result: True ##[section]Starting: AzureResourceManagerTemplateDeployment ============================================================================== Task : ARM template deployment Description : Deploy an Azure Resource Manager (ARM) template to all the deployment scopes Version : 3.184.1 Author : Microsoft Corporation Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment ============================================================================== ##[debug]Using node path: /home/vsts/agents/2.183.1/externals/node10/bin/node ##[debug]agent.TempDirectory=/home/vsts/work/_temp ##[debug]loading inputs and endpoints ##[debug]loading INPUT_DEPLOYMENTSCOPE ##[debug]loading INPUT_CONNECTEDSERVICENAME ##[debug]loading INPUT_SUBSCRIPTIONNAME ##[debug]loading INPUT_ACTION ##[debug]loading INPUT_LOCATION ##[debug]loading INPUT_TEMPLATELOCATION ##[debug]loading INPUT_CSMFILE ##[debug]loading INPUT_CSMPARAMETERSFILE ##[debug]loading INPUT_DEPLOYMENTMODE ##[debug]loading INPUT_ADDSPNTOENVIRONMENT ##[debug]loading ENDPOINT_AUTH_5e561276-7456-4ec2-8a13-e5394c4179a3 ##[debug]loading ENDPOINT_AUTH_SCHEME_5e561276-7456-4ec2-8a13-e5394c4179a3 ##[debug]loading ENDPOINT_AUTH_PARAMETER_5e561276-7456-4ec2-8a13-e5394c4179a3_TENANTID ##[debug]loading ENDPOINT_AUTH_PARAMETER_5e561276-7456-4ec2-8a13-e5394c4179a3_SERVICEPRINCIPALID ##[debug]loading ENDPOINT_AUTH_PARAMETER_5e561276-7456-4ec2-8a13-e5394c4179a3_AUTHENTICATIONTYPE ##[debug]loading ENDPOINT_AUTH_PARAMETER_5e561276-7456-4ec2-8a13-e5394c4179a3_SERVICEPRINCIPALKEY ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN ##[debug]loaded 20 ##[debug]Agent.ProxyUrl=undefined ##[debug]Agent.CAInfo=undefined ##[debug]Agent.ClientCert=undefined ##[debug]Agent.SkipCertValidation=undefined ##[debug]agent.proxyurl=undefined ##[debug]VSTS_ARM_REST_IGNORE_SSL_ERRORS=undefined ##[debug]AZURE_HTTP_USER_AGENT=VSTS_501c23d7-fd7d-4eaf-aaff-846cc96306bf_build_4_0 ##[debug]check path : /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]adding resource file: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]system.culture=en-US ##[debug]Agent.TempDirectory=/home/vsts/work/_temp ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]agent.proxyurl=undefined ##[debug]VSTS_ARM_REST_IGNORE_SSL_ERRORS=undefined ##[debug]AZURE_HTTP_USER_AGENT=VSTS_501c23d7-fd7d-4eaf-aaff-846cc96306bf_build_4_0 ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]Setting resource path to /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/task.json ##[debug]check path : /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/task.json ##[debug]adding resource file: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/task.json ##[debug]system.culture=en-US ##[debug]Resource file has already set to: /home/vsts/work/_tasks/AzureResourceManagerTemplateDeployment_94a74903-f93f-4075-884f-dc11f34058b4/3.184.1/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/module.json ##[debug]deploymentScope=Subscription ##[debug]ConnectedServiceName=5e561276-7456-4ec2-8a13-e5394c4179a3 ##[debug]Processed: ##vso[telemetry.publish area=TaskEndpointId;feature=AzureResourceManagerTemplateDeployment]{"endpointId":"5e561276-7456-4ec2-8a13-e5394c4179a3"} (node:1534) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead. ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data armManagementPortalUrl = https://portal.azure.com/ ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data ScopeLevel = Subscription ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param scope = undefined ARM Service Connection deployment scope - Subscription ##[debug]subscriptionName=40f2d467-c27b-4ded-a7bc-82dc0b95cbb2 ##[debug]resourceGroupName=undefined ##[debug]action=Create Or Update Resource Group ##[debug]location=eastus ##[debug]deploymentMode=Incremental ##[debug]templateLocation=Linked artifact ##[debug]csmFile=/home/vsts/work/1/s/poc.json ##[debug]csmParametersFile=/home/vsts/work/1/s ##[debug]overrideParameters=undefined ##[debug]outputVariable=undefined ##[debug]deploymentName=undefined ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth scheme = ServicePrincipal ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data subscriptionid = 40f2d467-c27b-4ded-a7bc-82dc0b95cbb2 ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data subscriptionname = Visual Studio Professional Subscription ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param serviceprincipalid = *** ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data environmentAuthorityUrl = https://login.windows.net/ ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param tenantid = *** ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3=https://management.azure.com/ ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data environment = AzureCloud ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth scheme = ServicePrincipal ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data msiclientId = undefined ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data activeDirectoryServiceEndpointResourceId = https://management.core.windows.net/ ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data AzureKeyVaultServiceEndpointResourceId = https://vault.azure.net ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data AzureKeyVaultDnsSuffix = vault.azure.net ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data ScopeLevel = Subscription ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param authenticationType = *** ##[debug]credentials spn endpoint ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param serviceprincipalkey = *** ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data EnableAdfsAuthentication = false ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param apitoken = undefined ##[debug]{"subscriptionID":"40f2d467-c27b-4ded-a7bc-82dc0b95cbb2","subscriptionName":"Visual Studio Professional Subscription","servicePrincipalClientID":"***","environmentAuthorityUrl":"https://login.windows.net/","tenantID":"***","url":"https://management.azure.com/","environment":"AzureCloud","scheme":"ServicePrincipal","activeDirectoryResourceID":"https://management.azure.com/","azureKeyVaultServiceEndpointResourceId":"https://vault.azure.net","azureKeyVaultDnsSuffix":"vault.azure.net","scopeLevel":"Subscription","authenticationType":"***","servicePrincipalKey":***,"isADFSEnabled":false,"applicationTokenCredentials":{"clientId":"***","domain":"***","baseUrl":"https://management.azure.com/","authorityUrl":"https://login.windows.net/","activeDirectoryResourceId":"https://management.azure.com/","isAzureStackEnvironment":false,"authType":"***","secret":***,"isADFSEnabled":false}} ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth scheme = ServicePrincipal ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth scheme = ServicePrincipal ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data subscriptionid = 40f2d467-c27b-4ded-a7bc-82dc0b95cbb2 ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data subscriptionname = Visual Studio Professional Subscription ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param serviceprincipalid = *** ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data environmentAuthorityUrl = https://login.windows.net/ ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param tenantid = *** ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3=https://management.azure.com/ ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data environment = AzureCloud ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth scheme = ServicePrincipal ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data msiclientId = undefined ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data activeDirectoryServiceEndpointResourceId = https://management.core.windows.net/ ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data AzureKeyVaultServiceEndpointResourceId = https://vault.azure.net ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data AzureKeyVaultDnsSuffix = vault.azure.net ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data ScopeLevel = Subscription ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data graphUrl = https://graph.windows.net/ ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param authenticationType = *** ##[debug]credentials spn endpoint ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param serviceprincipalkey = *** ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 data EnableAdfsAuthentication = false ##[debug]5e561276-7456-4ec2-8a13-e5394c4179a3 auth param apitoken = undefined ##[debug]{"subscriptionID":"40f2d467-c27b-4ded-a7bc-82dc0b95cbb2","subscriptionName":"Visual Studio Professional Subscription","servicePrincipalClientID":"***","environmentAuthorityUrl":"https://login.windows.net/","tenantID":"***","url":"https://management.azure.com/","environment":"AzureCloud","scheme":"ServicePrincipal","activeDirectoryResourceID":"https://graph.windows.net/","azureKeyVaultServiceEndpointResourceId":"https://vault.azure.net","azureKeyVaultDnsSuffix":"vault.azure.net","scopeLevel":"Subscription","authenticationType":"***","servicePrincipalKey":***,"isADFSEnabled":false,"applicationTokenCredentials":{"clientId":"***","domain":"***","baseUrl":"https://management.azure.com/","authorityUrl":"https://login.windows.net/","activeDirectoryResourceId":"https://graph.windows.net/","isAzureStackEnvironment":false,"authType":"***","secret":***,"isADFSEnabled":false}} ##[debug]deploymentOutputs=undefined ##[debug]addSpnToEnvironment=false ##[debug]Processed: ##vso[telemetry.publish area=TaskDeploymentMethod;feature=AzureResourceManagerTemplateDeployment]{"deploymentScope":"Subscription","deploymentMode":"Incremental"} Creating deployment parameters. ##[debug]System.DefaultWorkingDirectory=/home/vsts/work/1/s ##[debug]defaultRoot: '/home/vsts/work/1/s' ##[debug]findOptions.allowBrokenSymbolicLinks: 'false' ##[debug]findOptions.followSpecifiedSymbolicLink: 'true' ##[debug]findOptions.followSymbolicLinks: 'true' ##[debug]matchOptions.debug: 'false' ##[debug]matchOptions.nobrace: 'true' ##[debug]matchOptions.noglobstar: 'false' ##[debug]matchOptions.dot: 'true' ##[debug]matchOptions.noext: 'false' ##[debug]matchOptions.nocase: 'false' ##[debug]matchOptions.nonull: 'false' ##[debug]matchOptions.matchBase: 'false' ##[debug]matchOptions.nocomment: 'false' ##[debug]matchOptions.nonegate: 'false' ##[debug]matchOptions.flipNegate: 'false' ##[debug]pattern: '/home/vsts/work/1/s/poc.json' ##[debug]findPath: '/home/vsts/work/1/s/poc.json' ##[debug]statOnly: 'true' ##[debug]found 1 paths ##[debug]applying include pattern ##[debug]1 matches ##[debug]1 final results ##[debug]Loading CSM Template File.. /home/vsts/work/1/s/poc.json ##[debug]Detecting file encoding using BOM ##[debug]Unable to detect File encoding using BOM ##[debug]Detecting file encoding without BOM The detected encoding for file '/home/vsts/work/1/s/poc.json' is 'utf-8' ##[error]Check out the troubleshooting guide to see if your issue is addressed: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting ##[debug]Processed: ##vso[task.issue type=error;]Check out the troubleshooting guide to see if your issue is addressed: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting ##[debug]task result: Failed ##[error]Error: Ensure the Template file ( '/home/vsts/work/1/s/poc.json' ) is valid. Task failed while parsing with following error: Unexpected token ##[debug]Processed: ##vso[task.issue type=error;]Error: Ensure the Template file ( '/home/vsts/work/1/s/poc.json' ) is valid. Task failed while parsing with following error: Unexpected token %0A in JSON at position 321 ##[debug]Processed: ##vso[task.complete result=Failed;]Error: Ensure the Template file ( '/home/vsts/work/1/s/poc.json' ) is valid. Task failed while parsing with following error: Unexpected token %0A in JSON at position 321 ##[section]Finishing: AzureResourceManagerTemplateDeployment ```
artisticcheese commented 3 years ago

Same with comments. It fails if there are comments in document which are not allowed in JSON but allowed in JSONC

anuragc617 commented 3 years ago

Hi @SDedik and @artisticcheese yes, you are right. This is the limitation of task as for parsing and validation csm file we are using javascript JSON.parse which expects valid JSON. The multiline string and comments is supported in the latest version of ARM templates as mentioned in this doc that you must use the --handle-extended-json-format switch in Azure CLI version above 2.3.0

SDedik commented 3 years ago

@anuragc617, the root cause of the failure is well understood, as we as possible workarounds like switching to AzureCLI task or AzurePowerShell task. However... Think of it this way. I have an ARM template file which can be fed to the Azure Resource Manager via the Azure Portal, Azure API, Azure PowerShell and Azure CLI and it works as intended with all of the above. But, when I try to use the AzureResourceManagerTemplateDeployment task, the task fails. Do you acknowledge that this is a valid use-case and with such the behavior of the AzureResourceManagerTemplateDeployment task is a bug? What is the correct way of addressing this bug is a next question, but the fact that it needs to be addressed is of no doubt.

Also, you made a mistake in your post here:

you must use the --handle-extended-json-format switch in Azure CLI version above 2.3.0

According to doc provided it should be Azure CLI version 2.3.0 and below. Thus emphasizing that current versions of Azure CLI supports templates with multiline strings natively without additional switches. To quote the doc:

To deploy templates with multi-line strings by using Azure CLI with version 2.3.0 or older, you must use the --handle-extended-json-format switch.

artisticcheese commented 3 years ago

In addition to that please at least provide better explanation for failures. It tooks me hours to figure out what the issue is since error message so non-descriptive

anuragc617 commented 3 years ago

@SDedik yes, we acknowledge that this is a bug in AzureResourceManagerTemplateDeployment task. We have started internal discussion on how to address this bug. Sorry, my bad, yes it is t should be Azure CLI version 2.3.0 and below
@artisticcheese we will look at it as well if we can provide better message while error in template parsing

jikuja commented 3 years ago

The linked documentation is really bad and does not describe that multiline JSON support is done on the client-side as a template preprocessing step.

That is the reason why deploying templates with multiline strings or comments works with Az CLI or with Powershell. The documentation should clearly state that other deployment methods will FAIL:

Every ADO task doing ARM deployment should document that multiline and JSON comments are unsupported if they are unsupported. Also, the main documentation site MUST clearly state that multiline and JSON comments are selected client-only features. The current wording is really misleading.

edit: to make thing even worse this task strips out JSON comments before deployment and therefore only part of the extended JSON features are supported and is fully undocumented.

artisticcheese commented 3 years ago

@jikuja You are saying that powershell cmdlet before submitting deployment to Azure backplane transpiles multi-line string into single line string?

jikuja commented 3 years ago

Might be. Not sure what is the state of the current code but clients usually requires special code to handle JSON comments and/or multiline strings if they are parsing template file instead of passing contents of a file as a string to API.

Requires more testing by end-users for end-users to understand the current situation OR serious documentation updates.

mmujtaba1 commented 1 year ago

Multiline strings are not supported if you are deploying through Azure portal, a DevOps pipeline, or the REST API https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/syntax#multi-line-strings