Invalid schema deploying Bicep Blueprint artifact with arm template

vho002asr commented 1 year ago

Bicep version 0.13.1

Describe the bug We expected to create a new blueprint artifact with bicep deploy of an arm template under a newly created resourcegroup also created in our blueprint deploy. However it fails to deploy a managed user identity template within a blueprint artifact.

To Reproduce Steps to reproduce the behavior: We deployed below bicep code with corresponding parameters on subscription scope. Which results in following error: Status Message: Required | properties must be provided: roleDefinitionId,principalIds or | policyDefinitionId.

According to Bicep documentation https://learn.microsoft.com/en-us/azure/templates/microsoft.blueprint/2018-11-01-preview/blueprints/artifacts for kind='template' we do not need any of the required attributes provided by above error message.

When we manually add an ARM template artifact under the resource group in the Blueprint portal it works just fine.

Additional context Our blueprint bicep code:

resource blueprint 'Microsoft.Blueprint/blueprints@2018-11-01-preview' = {
  name: blueprintName
  properties: {
    targetScope: 'subscription'
    description: 'Example text'
    resourceGroups: {
      ResourceGroup: {
        name: ResourceGroupName
        location: region
        metadata: {
          displayName: 'Blueprint RG'
        }
      }
    }
  }
}

resource UserAssignedIdenty 'Microsoft.Blueprint/blueprints/artifacts@2018-11-01-preview' = {
  name: CreateIdentity
  kind: 'template'
  parent: blueprint
  properties: {
    description: 'This will create a new System Assigned Identity.'
    displayName: 'Create Identity'
    parameters: {
      identityName: {
          value: '[parameters(\'identityName\')]'
      }
    }
    resourceGroup: ResourceGroupName
    template: any(identityTemplate)
  }
}

ARM template:

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "identityName": {
    "type": "string"
  },
  "location": {
    "type": "string",
    "defaultValue": "[resourceGroup().location]"
  }
  },
  "variables": {},
  "resources": [
      {
          "type": "Microsoft.ManagedIdentity/userAssignedIdentities",
          "apiVersion": "2018-11-30",
          "name": "[parameters('identityName')]",
          "location": "[parameters('location')]"
      }
  ],
  "outputs": {
      "principalId": {
          "type": "string",
          "value": "[reference(resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('identityName'))).principalId]"
      }
  }
}

alex-frankel commented 1 year ago

Can you share any correlation IDs from the error? Is it failing specifically on the creation of the template artifact? What happens if you try to create the blueprint without any artifacts?

vho002asr commented 1 year ago

The deployment 'DeployBlueprint-20221219T151208' failed with | error(s). Showing 1 out of 1 error(s). Status Message: Required | properties must be provided: roleDefinitionId,principalIds or | policyDefinitionId. (Code:InvalidSchema) CorrelationId: | 81213b5f-bce4-45fd-a0c0-12e0621705dcAt Line number: 194

It is indeed failing on the creation of the template artifact. We had succesful runs when we created a blueprint without any artifacts and blueprints with role assignment artifacts.

Although documentation and bicep linter says we should not provide roleDefinitionId, principalIds or policyDefinitionId when using template artifact it does seem to be working when we provide all 3 attributes. We'll do some more testing on that part today.

We also tested newer versions of the user assigned identity template, but same error response.

vho002asr commented 1 year ago

@alex-frankel did you have any time to look into this?

We tried to provide roleDefinitionId, principalIds and policyDefinitionId as well, resulting in following error: InvalidSchema: Path:properties.template, Schema:#/definitions/TemplateArtifactProperties/properties/template, Error: Invalid type. Expected Object but got String.

If the type is changed to Object a new error message pops up asking for a String instead of Object type. This seems to be quite contradicting.

alex-frankel commented 1 year ago

I am not sure what is happening. We will investigate.

If it is urgent, I would recommend opening a support ticket, which will have an SLA.

majastrz commented 1 year ago

I checked our logs and there was definitely the failure you mentioned for the correlation ID. The internal exception indicated a schema validation failure and didn't provide any information about the JSON that was sent on the request.

There are missing pieces in the provided Bicep code, so I tried to recreate something similar. My deployment succeeded without any issues. Here are the files I used.

main.bicep:

targetScope = 'subscription'

param ResourceGroupName string

resource bp 'Microsoft.Blueprint/blueprints@2018-11-01-preview' = {
  name: 'majastrz-bp'
  properties: {
    targetScope: 'subscription'
    description: 'My blueprint'
    resourceGroups: {
      ResourceGroup: {
        name: ResourceGroupName
        location: 'westus'
        metadata: {
          displayName: 'Blueprint RG'
        }
      }
    }
  }
}

resource artifact 'Microsoft.Blueprint/blueprints/artifacts@2018-11-01-preview' = {
  name: 'identity'
  parent: bp
  kind: 'template'
  properties: {
    parameters: {
      identityName: {
        value: '[parameters(\'identityName\')]'
      }
    }
    template: loadJsonContent('identity.json')
    resourceGroup: ResourceGroupName
  }
}

identity.json:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "metadata": {
    "_generator": {
      "name": "bicep",
      "version": "0.13.1.58284",
      "templateHash": "8672000923298299324"
    }
  },
  "parameters": {
    "identityName": {
      "type": "string"
    }
  },
  "resources": [
    {
      "type": "Microsoft.ManagedIdentity/userAssignedIdentities",
      "apiVersion": "2022-01-31-preview",
      "name": "[parameters('identityName')]",
      "location": "[resourceGroup().location]"
    }
  ]
}

@vho002asr Would you be able to provide a minimal repro that compiles in Bicep but still reproduces the issue you mentioned when you try to deploy it?

I also have the following questions about the Bicep code you provided:

How are you setting the value of identityTemplate variable?
In the artifact properties you have resourceGroup: ResourceGroupName. Shouldn't that be resourceGroup: 'ResourceGroup'?
In the artifact parameters you are specifying an expression for the identityName parameter in the template, pointing to the blueprint parameter with the same name, but the provided blueprint doesn't declare that parameter. Is that intentional?

vho002asr commented 1 year ago

Hi Marcin,

Thanks for your code, I tried it on our side and it ran successfully.

1 Previously we ran the identityTemplate as a parameter and also tried it as a separate json file. 2 Is a parameter with the name of the resourcegroup. 3 IdentityName, was for testing purposes.

Maybe an idea to add an example part to the documentation? template: loadJsonContent('pathtotemplate.json')

For us this resolves the support ticket, you have our gratitude!

Met vriendelijke groet,

Vincent Hommersen

[Beschrijving: asrnieuw] Data Engineer DPS Technology Services II a.s.r. Nederland M: 06 - 38723963 E: @.**@.>

From: Marcin Jastrzebski @.> Sent: zaterdag 7 januari 2023 03:15 To: Azure/bicep @.> Cc: Hommersen V.R. (Vincent) @.>; Mention @.> Subject: EXT: Re: [Azure/bicep] Invalid schema deploying Bicep Blueprint artifact with arm template (Issue #9350)

U ontvangt niet vaak e-mail van @.**@.>. Meer informatie over waarom dit belangrijk ishttps://aka.ms/LearnAboutSenderIdentification Deze mail komt van buiten a.s.r. Help mee om a.s.r. veilig te houden en denk na voordat je op een link klikt. Meld verdachte mails via de Hoxhunt knop.

I checked our logs and there was definitely the failure you mentioned for the correlation ID. The internal exception indicated a schema validation failure and didn't provide any information about the JSON that was sent on the request.

There are missing pieces in the provided Bicep code, so I tried to recreate something similar. My deployment succeeded without any issues. Here are the files I used.

main.bicep:

targetScope = 'subscription'

param ResourceGroupName string

resource bp @.***' = {

name: 'majastrz-bp'

properties: {

targetScope: 'subscription'

description: 'My blueprint'

resourceGroups: {

  ResourceGroup: {

    name: ResourceGroupName

    location: 'westus'

    metadata: {

      displayName: 'Blueprint RG'

    }

  }

}

}

resource artifact @.***' = {

name: 'identity'

parent: bp

kind: 'template'

properties: {

parameters: {

  identityName: {

    value: '[parameters(\'identityName\')]'

  }

}

template: loadJsonContent('identity.json')

resourceGroup: ResourceGroupName

}

identity.json:

{

"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

"contentVersion": "1.0.0.0",

"metadata": {

"_generator": {

  "name": "bicep",

  "version": "0.13.1.58284",

  "templateHash": "8672000923298299324"

}

},

"parameters": {

"identityName": {

  "type": "string"

}

},

"resources": [

{

  "type": "Microsoft.ManagedIdentity/userAssignedIdentities",

  "apiVersion": "2022-01-31-preview",

  "name": "[parameters('identityName')]",

  "location": "[resourceGroup().location]"

}

]

}

@vho002asrhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvho002asr&data=05%7C01%7Cvincent.hommersen%40asr.nl%7C38282f382135487cdeb508daf0550746%7C092ed8ead21743afbb33abd3c252103c%7C0%7C0%7C638086545233644198%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ouf5f8GAEauk%2Bd12H9fbDUMKfyQ8f8ZVtCq9MBGwDj0%3D&reserved=0 Would you be able to provide a minimal repro that compiles in Bicep but still reproduces the issue you mentioned when you try to deploy it?

I also have the following questions about the Bicep code you provided:

How are you setting the value of identityTemplate variable?
In the artifact properties you have resourceGroup: ResourceGroupName. Shouldn't that be resourceGroup: 'ResourceGroup'?
In the artifact parameters you are specifying an expression for the identityName parameter in the template, pointing to the blueprint parameter with the same name, but the provided blueprint doesn't declare that parameter. Is that intentional?

- Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbicep%2Fissues%2F9350%23issuecomment-1374351394&data=05%7C01%7Cvincent.hommersen%40asr.nl%7C38282f382135487cdeb508daf0550746%7C092ed8ead21743afbb33abd3c252103c%7C0%7C0%7C638086545233644198%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QCOmkg5DV%2BFvQk%2Fo8eEVXFTSYs0Kmc3fugTT1dTm6Rs%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FA43JYTTVOHXEWEANAIZ7GIDWRDGTRANCNFSM6AAAAAATDFHQ54&data=05%7C01%7Cvincent.hommersen%40asr.nl%7C38282f382135487cdeb508daf0550746%7C092ed8ead21743afbb33abd3c252103c%7C0%7C0%7C638086545233644198%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=akrZGkOJpQXZ0kJ%2FHNy1CcOfqqVfwsNFsxWDavj78v4%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.**@.>>

Azure / bicep

Invalid schema deploying Bicep Blueprint artifact with arm template #9350