[Question/Feedback]: Using AMBA without a Github Repo

[gerrynicol]

Check for previous/existing GitHub issues

• [X] I have checked for previous/existing GitHub issues

Description

Hi All,

I am trying to deploy AMBA using powershell from a local cloned repo set up with DevOps. When I try to deploy using the command below

New-AzManagementGroupDeployment -ManagementGroupId $pseudoRootManagementGroup -Location $location -TemplateFile ./patterns/alz/alzArm.json -TemplateParameterFile ./patterns/alz/alzArm.param.json

I am getting the error below. Are there limitations with deploying AMBA without using a Github Repo and deploying from either Azure Devops or locally form a cloned repo?

New-AzManagementGroupDeployment: 16:49:22 - Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'The template variable 'deploymentUris' is not valid: The language expression property 'templateLink' doesn't exist, available properties are 'template, templateHash, parameters, mode, provisioningState'.. Please see https://aka.ms/arm-functions for usage details.'. New-AzManagementGroupDeployment: The deployment validation failed

See attachment for further info.

[Brunoga-MS]

Hello @gerrynicol, thanks for your feedback. Looking at the description and the screenshot, I think the issue you run into was caused by the wrong URI in your command. The URI should always be in the form of a URL and it normally points to this path https://raw.githubusercontent.com/Azure/azure-monitor-baseline-alerts/main/patterns/alz/alzArm.json which contains the official code. You only need to change it to your repo and branch if you customized the code as per Customizing the AMBA policies.

Let me know if that helps.

Thanks, Bruno.

[gerrynicol]

Hi @Brunoga-MS

Thanks for getting back to me. Yes, I will need to customise the code. It would be good to get more information or a specific deployment guide for using Azure Devops, if indeed this is supported?

So far I have done as follows. Forked the repo to my own Github repo. Then taken a clone of the fork to Azure Devops. Looking at the guidance everything seems to suggest I have to use a URI in the forked github repo. When I try to use a URI in Azure Devops it doesnt work. The previous incaranation of the BAseline ALerts repo allowed me to refence a a local template file using New-AzManagementGroupDeployment -ManagementGroupId $pseudoRootManagementGroup -Location $location -TemplateFile.

[Brunoga-MS]

Hi @gerrynicol , the guidance for code customization is at Customizing the AMBA policies. It goes through the customization of current policies and does not cover how to add new policy. As far as the reference to local template, it is not possible, you can only reference local parameter file. If you need to deploy your local version, just commit your changes to your repo and put the link to your template.

Let me know if I can help more.

Thanks, Bruno.

[gerrynicol]

HI Bruno,

I'm happy with the customisation of the policies, thats explained well. Its the deployment options that I don't find clear. How can I deploy with an Azure Devops repo using the templateUri? Can this point at a private Azure Devops Repo and not just a Github repo (https://raw.githubusercontent.com). Can you give me example of how this is done? As previously mentioned, when I try to use a URI for the devops repo it doesnt work.

[Brunoga-MS]

Hi @gerrynicol, it should work as long as you pushed the changes to your remote Azure DevOps branch

[gerrynicol]

Hi @Brunoga-MS

I havent made any changes to push. I'm trying to see if there is alternative method of having to point to a github repo for the location of the alzArm.json file? I want the deployment to be done via Azure Devops. i.e a URL in Devops not github

As you can see in the attachment, Step 5 of the customizing-the-amba-policies still says that the deployment should point to a Github URL?

[Brunoga-MS]

Hi @gerrynicol , you could find an example at Deploying ARM Templates with Linked Templates from a Private Repository and references on how to deploy using private repos at Tutorial: Create a template spec with linked templates.

Hope that help.

Thanks, Bruno.

[alexeyzolotukhin]

I hit an identical issue. Ended up using the GitHub URI as pointed out in the docs, and my custom parameters file. It works, but I find it a bit strange that I have to link GitHub URL if I have forked AMBA repo into my local non-public DevOps repo. I guess it might work if my DevOps repo was public, but that's out of the question, which brings me to a feature request... Would it be possible to allow using -TemplateFile parameter instead of -TemplateUri and just pass a the local alzArm.json?

Thanks, Alex.

[arjenhuitema]

Thank you for your feedback @alexeyzolotukhin.

We can't use the -TemplateFile parameter as the ARM template uses linked templates. When referencing a linked template, the value of URI can't be a local file or a file that is only available on your local network. Azure Resource Manager must be able to access the template. This means they must be referenced using a URL which can be accessed from Azure (e.g. via a public GitHub repository)

An alternative is to use Template specs. Instead of maintaining your linked templates at an accessible endpoint, you can create a template spec that packages the main template and its linked templates into a single entity you can deploy. The template spec is a resource in your Azure subscription. It makes it easy to securely share the template with users in your organization. You use Azure role-based access control (Azure RBAC) to grant access to the template spec. This feature is currently in preview.

Reference to template specs: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/linked-templates?tabs=azure-powershell#template-specs

[NikolaiKleppe]

As mentioned the culprit of this error is because of the requirement that the linked template file has to be publicly accessible, which is easy with GitHub, but not so much with Azure DevOps.

We use Azure DevOps too and have solved it by uploading the required files (alzArm.json and every template URI it uses, see below) to a Storage Account during the pipeline itself, using SAS tokens. When changes are made with git it always pushes the new content to the storage account.

So the https://raw.githubusercontent.com/<...>/ url is then instead a blob url.

One challenge with this approach is that unfortunately the SAS authentication only works towards the main alzArm.json file, when this file then tries to call the other files in the Storage Account the SAS isn't passed along so authentication fails. Because of this we have to have the storage account container be publicly readable - But as these are just templates it's not that big of a deal

This also actually highlights that it isn't only the alzArm.json that has to be public, but every file referenced here, because of "templateLink.uri":

policies.json
DINE-ConnectivityAssignment.json
DINE-IdentityAssignment.json
DINE-LandingZoneAssignment.json
DINE-ManagementAssignment.json
DINE-ServiceHealthAssignment.json

[gerrynicol]

@arjenhuitema Thanks for that advice. I managed to get the default deployment to work using templatespecs. I have a high degree of customisation required after this, to change the code to use webhooks rather than email notifications and also creating multiple alert procesing rules per webhook. As @NikolaiKleppe has mentioned all of the linked templated need to be accessible publically.

@NikolaiKleppe . Would you mind sharing some further information on the steps you took to get the code into Azure Devops and also your process for keeping the devops repo up to date? For example, did you fork the repo into a personal/business github repo first?

[NikolaiKleppe]

@gerrynicol - Sure, If I understood your question correctly - It might not be optimal but basically I cloned AMBA directly (no fork) and pushed it to a new DevOps repository. Then I set an "upstream" remote towards this github, and the idea then is to pull changes when needed.

Something like this:

git remote add upstream https://github.com/Azure/azure-monitor-baseline-alerts.git
git fetch upstream
git checkout -b featureBranch
git merge (or rebase) upstream/main

So far I haven't had the need to pull any changes so I'm not 100% sure how well that works in practice right now.

Basically the same workflow as with forks

[gerrynicol]

Thanks @NikolaiKleppe. Apprecaite you taking the time to reply :). I have replicated that and seems to be a good option.

[lansalot]

This seems like an awful workaround tho - an Azure monitoring solution that requires a public github repository in order to work, or some hacky SAS/blob shenanigans?

Can MS not get this working so it works in their own hosted repositories? Surely, the advice can't be "use a public github repo to deploy enterprise architecture"? I appreciate there's little in terms of private info much beyond some management group names and perhaps an email address, but even so...