Azure / AzOps

AzOps is a PowerShell module which deploys (Push) ARM Resource Templates & Bicep files at all Azure scope levels and exports (Pull) ARM resource hierarchy.
https://aka.ms/AzOps
MIT License
371 stars 158 forks source link

Using AllowMultipleTemplateParameterFiles, when the suffix appears multiple times in the file path, the corresponding template file cannot be resolved #884

Closed Xitric closed 1 month ago

Xitric commented 1 month ago

Describe the bug

Currently, when AzOps resolves a template file from a parameter file, it uses two replace operations, one of which matches against the regex wildcard character ., rather than the literal character period (\.). Furthermore, by using two separate replace operations, the code misses a series of edge cases if the suffix appears multiple times in the template path. This is technically two separate issues.

Firstly, because AzOps matches against the wildcard character ., a parameter file templatexabc.xabc.bicepparam would try to locate a template file templat.bicep, because both exabc and .xabc match the regex .xabc. Using the regex \.xabc would correctly resolve templatexabc.bicep.

Secondly, because AzOps uses two replace operations, one for the suffix and another for the file extension, a parameter file template.xabc.xabc.bicepparam would try to locate a template file template.bicep, since both occurrences of .xabc are replaced with an empty string. This at least surprised my team, and I don't think this was ever intended behaviour. Only the last occurrence of .xabc should be removed.

We observed this issue since we use a Suffix regex \w+ and our suffixes are sometimes of the form 01 or 02. This ended up removing sections of our subscription and management group GUIDs which contain the character sequences 01 and 02:

root/mg (c9829f35-e20f-5ace-bf85-8dc91dfa66c5)/sub (b87a58f9-f7d7-4633-8c93-59c558df015e)/app.westeurope.01.bicepparam -> root/mg (c9829f35-e20f-5ace-bf85-8dc91dfa66c5)/sub (b87a58f9-f7d7-4633-8c93-59c558d5e)/app.westeurope.bicep

Steps to reproduce

  1. Enable AzOps.Core.AllowMultipleTemplateParameterFiles
  2. Create a PR adding a template file named templatexabc.xabc.bicep and a valid parameter file templatexabc.xabc.xabc.bicepparam
  3. Run Invoke-AzOpsPush and observe the error message

Screenshots

Output from our push pipeline:

[15:46:56][Resolve-ArmFileAssociation] Found AllowMultipleTemplateParameterFile root/mg (c9829f35-e20f-5ace-bf85-8dc91dfa66c5)/sub (b87a58f9-f7d7-4633-8c93-59c558df015e)/app.westeurope.01.bicepparam
[15:46:56][Resolve-ArmFileAssociation] Did NOT find template  for parameters root/mg (c9829f35-e20f-5ace-bf85-8dc91dfa66c5)/sub (b87a58f9-f7d7-4633-8c93-59c558df015e)/app.westeurope.01.bicepparam
[15:46:56][Resolve-ArmFileAssociation] Determining template from main template file: /usr/local/share/powershell/Modules/AzOps/2.6.3/data/template/template.json
ConvertFrom-Json: Conversion from JSON failed with error: Error parsing undefined value. Path '',
line 1, position 1.
Jefajers commented 1 month ago

Hi @Xitric, thanks for reporting this issue and offering a solution to this issue.

I have reproduced the issue and agree the current behavior is neither the indented behavior (possibility of matching wrong places of the string with regex) and ironically the json and bicep logic contains a slight difference on the second operation where one of them caters for the regex behavior \. however this has been overlook in the current codebase at the other lines of code in combination with dual regex operation the code can perform the work in one operation.

We will review the PR and plan for a release with the suggested fixes 🥇.