Benefit

Tags with hidden- prefix are not visible in the Azure portal except in the JSON view. by adding the additional hidden tags for CARML module names and versions allows customers to:

Collect internal usage and inventory data for the consumption of the module. When a module requires update, customers can easily find out which resources need to be updated using simple Azure Resource Graph (ARG) queries.
Provides the ability to control the usage of -prerelease module versions being deployed outside of testing environments using Azure Policy

I am happy to raise PR to implement this feature since @ahmadabdalla and I have already developed this feature in a customer's environment.

How to implement

I have implemented this feature in a customer's environment. here's what I did:

update the pipeline tasks to write the full major.minor.patch version number to the version.json in the run time, before the module publishing step.
update each CARML module to add the module name and version. version number is extracted from version.json file:

var moduleVersion = loadJsonContent('./version.json').version

var mergedTags = union(tags, {
    'hidden-module_name': 'storage/storage-account'
    'hidden-module_version': moduleVersion
  })

Pass the mergedTags to the tags property of the resources

How to consume the hidden module name and version tags

Collecting internal usage telemetry data

Here are some sample ARG queries:

Get all Overlay module usage

resources
| where tags['hidden-module_name']  matches regex '.'
| summarize count() by type

Get all Key Vaults deployed using Overlay module and show version

resources
| where type =~ "microsoft.keyvault/vaults"
| where tags['hidden-module_name'] =~ 'key-vault/vault'
| project name, tags
| mvexpand tags
| extend tagKey = tostring(bag_keys(tags)[0])
| extend tagValue = tostring(tags[tagKey])
| distinct name, tagKey, tagValue
| where tagKey =~ "hidden-module_version"
| project KeyVaultName = name, OverlayModuleVersion = tagValue

Restrict resources to be deployed using prerelease overlay module versions

Here's a sample policy definition I have created to restrict the use of `-prerelease' versions:

{
  "name": "pol-restrict-prerelease-overlay-module-versions",
  "properties": {
    "displayName": "Restrict resources to be deployed using prerelease overlay module versions",
    "description": "Prerelease module versions are published for testing purposes only. They are not intended for production use and they have not gone through code review and validation. This policy restricts resources from being deployed using prerelease overlay module versions.",
    "metadata": {
      "category": "Code Vulnerability",
      "version": "1.0.0",
      "preview": false,
      "depreciated": false
    },
    "mode": "Indexed",
    "parameters": {
      "effect": {
        "type": "String",
        "metadata": {
          "displayName": "Effect",
          "description": "Enable or disable the execution of the policy"
        },
        "allowedValues": [
          "Audit",
          "Deny",
          "Disabled"
        ],
        "defaultValue": "Deny"
      }
    },
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "tags[hidden-module_version]",
            "exists": true
          },
          {
            "field": "tags[hidden-module_version]",
            "like": "*-prerelease"
          }
        ]
      },
      "then": {
        "effect": "[parameters('effect')]"
      }
    }
  }
}

AlexanderSehr commented 3 months ago

There are a few things we could consider for AVM such as

Instead of loading the version.json (with only major.minor) take the actual version during publishing like

  tags: {
    'hidden-version': replace('-..--..-', '.', '-')
  }

Have an input parameter (bool) that allows you to control if you want this or not
etc.

microsoft-github-policy-service[bot] commented 3 months ago

[!IMPORTANT] The "Needs: Triage :mag:" label must be removed once the triage process is complete!

[!TIP] For additional guidance on how to triage this issue/PR, see the BRM Issue Triage documentation.

microsoft-github-policy-service[bot] commented 2 months ago

[!WARNING] Tagging the AVM Core Team (@Azure/avm-core-team-technical-bicep) due to a module owner or contributor having not responded to this issue within 3 business days. The AVM Core Team will attempt to contact the module owners/contributors directly.

[!TIP]

To prevent further actions to take effect, the "Status: Response Overdue 🚩" label must be removed, once this issue has been responded to.

To avoid this rule being (re)triggered, the ""Needs: Triage :mag:" label must be removed as part of the triage process (when the issue is first responded to)!

microsoft-github-policy-service[bot] commented 2 months ago

[!WARNING] Tagging the AVM Core Team (@Azure/avm-core-team-technical-bicep) due to a module owner or contributor having not responded to this issue within 3 business days. The AVM Core Team will attempt to contact the module owners/contributors directly.

[!TIP]

To prevent further actions to take effect, the "Status: Response Overdue 🚩" label must be removed, once this issue has been responded to.

To avoid this rule being (re)triggered, the ""Needs: Triage :mag:" label must be removed as part of the triage process (when the issue is first responded to)!

microsoft-github-policy-service[bot] commented 2 months ago

[!CAUTION] This issue requires the AVM Core Team's (@Azure/avm-core-team-technical-bicep) immediate attention as it hasn't been responded to within 6 business days.

[!TIP]

To avoid this rule being (re)triggered, the "Needs: Triage :mag:" and "Status: Response Overdue :triangular_flag_on_post:" labels must be removed when the issue is first responded to!

Remove the "Needs: Immediate Attention :bangbang:" label once the issue has been responded to.

microsoft-github-policy-service[bot] commented 2 months ago

[!WARNING] Tagging the AVM Core Team (@Azure/avm-core-team-technical-bicep) due to a module owner or contributor having not responded to this issue within 3 business days. The AVM Core Team will attempt to contact the module owners/contributors directly.

[!TIP]

To prevent further actions to take effect, the "Status: Response Overdue 🚩" label must be removed, once this issue has been responded to.

To avoid this rule being (re)triggered, the ""Needs: Triage :mag:" label must be removed as part of the triage process (when the issue is first responded to)!

microsoft-github-policy-service[bot] commented 2 months ago

[!CAUTION] This issue requires the AVM Core Team's (@Azure/avm-core-team-technical-bicep) immediate attention as it hasn't been responded to within 6 business days.

[!TIP]

To avoid this rule being (re)triggered, the "Needs: Triage :mag:" and "Status: Response Overdue :triangular_flag_on_post:" labels must be removed when the issue is first responded to!

Remove the "Needs: Immediate Attention :bangbang:" label once the issue has been responded to.

microsoft-github-policy-service[bot] commented 2 months ago

[!WARNING] Tagging the AVM Core Team (@Azure/avm-core-team-technical-bicep) due to a module owner or contributor having not responded to this issue within 3 business days. The AVM Core Team will attempt to contact the module owners/contributors directly.

[!TIP]

To prevent further actions to take effect, the "Status: Response Overdue 🚩" label must be removed, once this issue has been responded to.

To avoid this rule being (re)triggered, the ""Needs: Triage :mag:" label must be removed as part of the triage process (when the issue is first responded to)!

microsoft-github-policy-service[bot] commented 2 months ago

[!CAUTION] This issue requires the AVM Core Team's (@Azure/avm-core-team-technical-bicep) immediate attention as it hasn't been responded to within 6 business days.

[!TIP]

To avoid this rule being (re)triggered, the "Needs: Triage :mag:" and "Status: Response Overdue :triangular_flag_on_post:" labels must be removed when the issue is first responded to!

Remove the "Needs: Immediate Attention :bangbang:" label once the issue has been responded to.

microsoft-github-policy-service[bot] commented 2 months ago

[!CAUTION] This issue requires the AVM Core Team's (@Azure/avm-core-team-technical-bicep) immediate attention as it hasn't been responded to within 6 business days.

[!TIP]

To avoid this rule being (re)triggered, the "Needs: Triage :mag:" and "Status: Response Overdue :triangular_flag_on_post:" labels must be removed when the issue is first responded to!

Remove the "Needs: Immediate Attention :bangbang:" label once the issue has been responded to.

Azure / bicep-registry-modules

[Feature Request]: Introduce hidden tags to store module name and version #2503

Description

Benefit

How to implement

How to consume the hidden module name and version tags

Collecting internal usage telemetry data

Get all Overlay module usage

Get all Key Vaults deployed using Overlay module and show version

Restrict resources to be deployed using prerelease overlay module versions