Unable to build Azure DevOps pipeline to assert CAF pre-flight rule (using bicep file)

[majorku5anagi]

Hi,

I'm editing my original question as it seems that I failed to differentiate the pre/in-flight settings and my question diverged from where it should really land so the post became a mess... To recap... I'm not sure on how to deploy CAF pre-flight check on Azure Pipelines. I've made simple stage with several tasks, first one being building the ARM template from my main.bicep file and continuing with the rest of the code that was copied and pasted from the PSRule.Rules.CAF section with yaml pre-flight example. The result is 0 processed rules.

trigger:
  branches:
    include:
    - Development
pool:
  vmImage: ubuntu-latest
variables:
- name: ResourceGroupName
  value: 'biceptesting'
stages:

- stage: Cloud_Adoption_FRWK
  jobs:

#
# STEP 2: Template validation
#

  - job: 'analyze_arm'
    displayName: 'Analyze templates'
    pool:
      vmImage: 'ubuntu-18.04'
    steps:
    - task: CmdLine@2
      name: BuildARMfromBicep
      displayName: Build ARM from Bicep
      inputs:
        script: |
          az bicep build --file deploy/main.bicep
    # STEP 3: Install PSRule.Rules.CAF from the PowerShell Gallery
    - task: ps-rule-install@0
      displayName: Install PSRule.Rules.CAF
      inputs:
        module: 'PSRule.Rules.CAF'   # Install PSRule.Rules.CAF from the PowerShell Gallery.

    # STEP 4: Export template data for analysis
    - powershell: Get-AzRuleTemplateLink | Export-AzTemplateRuleData -OutputPath 'out/templates/';
      displayName: 'Export template data'

    # STEP 5: Run analysis against exported data
    - task: ps-rule-assert@0
      displayName: Analyze Azure template files
      inputs:
        inputType: inputPath
        inputPath: 'out/templates/'   # Read objects from JSON files in 'out/templates/'.
        modules: 'PSRule.Rules.CAF'   # Analyze objects using the rules within the PSRule.Rules.CAF PowerShell module.

This is how it looks when in-depth checking of the run>

Export part (this looks odd too):

Assert rule part:

[BernieWhite]

@majorku5anagi Thanks for raising the issue. The examples are a little out of date. Let me update them.

To answer your question.

Export template data is not returning any output. This is because Get-AzRuleTemplateLink looks for JSON parameter files.

The PSRule.Rules.CAF module works mostly the same as a companion to PSRule.Rules.Azure.

You could update you pipeline to something like:

# STEP 4: Export template data for analysis
- powershell: Export-AzRuleTemplateData -TemplateFile deploy/main.json -OutputPath 'out/templates/';
  displayName: 'Export template data'

---

My suggestions would be to:

• Bump ps-rule-install@0 to ps-rule-install@1 and ps-rule-assert@0 to ps-rule-assert@1 because @0 uses a non-supported version of Node.
• Set the AZURE_BICEP_FILE_EXPANSION: true option in ps-rule.yaml. If you expand this way Install PSRule.Rules.CAF and Export template data is not required.

It is the same instructions as PSRule for Azure, just using PSRule.Rules.CAF instead. You can even do this is you are running both:

# Analyze Azure resources using PSRule for Azure
- task: ps-rule-assert@1
  displayName: Analyze Azure template files
  inputs:
    inputType: repository
    modules: 'PSRule.Rules.Azure,PSRule.Rules.CAF'

https://azure.github.io/PSRule.Rules.Azure/creating-your-pipeline/ https://azure.github.io/PSRule.Rules.Azure/using-bicep/

Hope that helps.

[majorku5anagi]

@BernieWhite thanks for the update. I tried the second time, this time guided by your comment about the fact that both, PSRule.Rules.Azure & PSRule.Rules.CAF should use the same instructions and had different results depending on whether I used them in single or separate stages. When I do this in one stage like in your example and perform one-liner module call for both PSRule.Rules.Azure & PSRule.Rules.CAF everything works fine. So this basically resolves my issue and thanks for that! 👍

Can PSRule.Rules.CAF work in separate stage independently? Because PSRule.Rules.Azure part of the pipeline works fine I tried to just duplicate it in a new stage and changed it so the module call is for PSRule.Rules.CAF (I know its an overkill but I want them to be separated). Mentioned ps-rule.yaml file already had been set with bicep expansion (see below), I'm not sure if I need to set some additional configurations there so that they can be run independently in separate stages?

PSRule.Rules.Azure & PSRule.Rules.CAF stages:

- stage: AZ_Well_Archt_FRW
  jobs: 
  - job: AZ_Well_Arch_Framework 
    displayName: Azure Well-Architected Framework Guidelines
    steps:
    - task: ps-rule-assert@1
      continueOnError: true
      inputs:
        inputType: 'inputPath'
        inputPath: 'deploy/main.bicep'
        modules: 'PSRule.Rules.Azure'
        outputFormat: 'NUnit3'
        outputPath: '$(Build.ArtifactStagingDirectory)/report/awafk-ps-rule-results.xml'
    - task: PublishTestResults@2
      continueOnError: true
      inputs:
        testResultsFormat: 'NUnit'
        testResultsFiles: '$(Build.ArtifactStagingDirectory)/report/awafk-ps-rule-results.xml'
        testRunTitle: 'Azure Well-Architected Framework Test'
        buildPlatform: 'Bicep'
    - task: PublishBuildArtifacts@1
      continueOnError: true
      inputs:
        pathToPublish: '$(Build.ArtifactStagingDirectory)/report'
        artifactName: 'AWAF_Report'

- stage: AZ_Cloud_Adpt_FRW
  jobs: 
  - job: AZ_Cloud_Adoption_Framework 
    displayName: Azure Cloud Adoption Framework Guidelines
    steps:
    - task: ps-rule-assert@1
      continueOnError: true
      inputs:
        inputType: 'inputPath'
        inputPath: 'deploy/main.bicep'
        modules: 'PSRule.Rules.CAF'
        outputFormat: 'NUnit3'
        outputPath: '$(Build.ArtifactStagingDirectory)/report/acaf-ps-rule-results.xml'
    - task: PublishTestResults@2
      continueOnError: true
      inputs:
        testResultsFormat: 'NUnit'
        testResultsFiles: '$(Build.ArtifactStagingDirectory)/report/acaf-ps-rule-results.xml'
        testRunTitle: 'Azure Cloud Adoption Framework Test'
        buildPlatform: 'Bicep'
    - task: PublishBuildArtifacts@1
      continueOnError: true
      inputs:
        pathToPublish: '$(Build.ArtifactStagingDirectory)/report'
        artifactName: 'ACAF_Report'

ps-rule.yaml:

configuration:
  AZURE_BICEP_FILE_EXPANSION: true

rule:
  # Enable custom rules that don't exist in the baseline
  includeLocal: true

execution:
  notProcessedWarning: false

output:
  culture:
  - en-US

[BernieWhite]

@majorku5anagi Good question. You absolutely should be able to run PSRule.Rules.CAF by itself in a separate job.

I think the main issue you are getting here is that the dependency chain of PSRule.Rules.CAF v0.3.0 currently only required v1.9.1 of PSRule.Rules.Azure as you can see in your output. Bicep support was experimental prior to v1.11.0, so that is probably related.

Try adding an explicit install task for PSRule.Rules.Azure which should install the latest version.

- task: ps-rule-install@1
  displayName: Install latest PSRule.Rules.Azure
  inputs:
    module: 'PSRule.Rules.Azure'

The next release on PSRule for Cloud Adoption Framework should address this.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs within 7 days. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs within 7 days. Thank you for your contributions.

[github-actions[bot]]

This issue was closed because it has not had any recent activity.