Add ability to specify an existing Resource Group and deploy to RG scope

[colbylwilliams]

• [x] Make sure you've installed the latest version using instructions in the wiki

Output from az dev version Run azd version and copy and paste the output here: azd version 0.0.1-beta.1530657 (commit bc9c19d56f2c3829d7ace2fbed54604cf64bad52)

Describe the bug Currently all templates are deployed at the subscription scope because they create a resource group on behalf of the user. Is there a plan to be able to pass in an existing resource group and have the template's bicep templates deploy directly to that resource group?

Often in the enterprise (where az dev could be extremely valuable) the user won't have contributor access to an entire subscription. Instead they will be given a resource group in a subscription managed by the central IT team.

If this is not already on the roadmap, please consider adding the concept of scope that would allow me to indicate that my template is either subscription scoped or resource group scoped.

If the template is resource group scoped, I would choose an existing resource group, which would be used during provisioning/deployment.

This could potentially be inferred from bicep template as it has the targetScope property, but I think it makes more sense to be in the azure.yaml.

[colbylwilliams]

cc @jongio @markweitzel

[jongio]

They could specify the group name or modify the bicep template to use 'existing' (bicep keyword).

In main.bicep

resource resourceGroup 'Microsoft.Resources/resourceGroups@2020-06-01' = {
  name: '${name}rg'
  location: location
}

To:

resource resourceGroup 'Microsoft.Resources/resourceGroups@2020-06-01' existing = {
  name: '${name}rg'
}

[ellismg]

Some related discussion as to what this might look like in in https://github.com/Azure/azure-dev/issues/963#issuecomment-1111496224.

I'm not sure that moving to the existing pattern in bicep would be sufficient, since it would still require the user doing the provision to have access to do a deployment at the subscription level, which may not be possible in larger organizations (does that sound correct to you, @colbylwilliams?)

FWIW - in Azure/azure-dev#373 we thought this might be an issue - so It's good to get some additional data from folks that this could be an issue.

[colbylwilliams]

Thanks for the context, and yes the existing keyword in the bicep template could be a good workaround for some, but isn't sufficient for what I'm asking.

Large organizations often have very specific naming conventions so we can't assume that the developer has any say in what the name of the resource group will be.

In addition, in some organizations the developer will not only get the resource group, but they'll also get a service principal that only has access to that resource group to be used with CI/CD.

So in large enterprises, it's safe to assume that the development team only has permissions on the resource group, and doesn't have any influence on what the name of the resource group will be. However, on the flipside, large organizations are trying to figure out how to provide a catalog of approved/compliant templates for development teams to start with, so this is a tool that could be extremely valuable if we can specify the scope as a resource group.

Side note: I'm also working with another product group within Microsoft that's building a tool that, among other things, automatically provisions the resource group for the developer. I'm trying to get this to work with that product, but I can guarantee in this scenario users won't have subscription level access, only permissions on the specific resource group (and no way to influence the name of the RG). This is a little outside the scope of this specific issue, but feel free to reach out if you want more context. Alias colbyw

[jongio]

Let's consider how we can build this switch from sub to rg deployments with iac only. We need to be very careful about mixing declarative and imperative operations as we need a source of truth for deployments and allowing run-time specification of things that will be deployed combines the two approaches.

Let's see if we can configure the bicep template to be either sub or group.

Principles

1. We do not want the CLI to create resources other than going through deployment commands in the az cli. This is so the bicep files + env vars are the source of truth
2. We want to streamline common scenarios and not prompt for things that aren't absolutely necessary, but allow the user/org modify templates to suit their needs.

User Restrictions

Let's detail out the scenarios we need to support:

1. User has to deploy resources to an existing resource group
2. User has to adhere to specific naming requirements
3. User doesn't have subscription level permissions
4. User isn't allow to create a new rg

How to support resource group overrides

Support ability for user to add resourceGroup parameter to main.parameters.json

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "name": {
      "value": "${AZURE_ENV_NAME}"
    },
    "location": {
      "value": "${AZURE_LOCATION}"
    },
    "principalId": {
      "value": "${AZURE_PRINCIPAL_ID}"
    },
    "resourceGroup": {
      "value": "${AZURE_RESOURCE_GROUP}"
    }
  }
}

Update resourceGroup.name from:

resource resourceGroup 'Microsoft.Resources/resourceGroups@2020-06-01' = {
  name: '${name}rg'
  location: location
}

To this to consider that new param.

resource resourceGroup 'Microsoft.Resources/resourceGroups@2020-06-01' = {
  name: empty(resourceGroup) ? '${name}rg' : resourceGroup
  location: location
}

Allow user to specify resource group as command-line argument, but do not prompt for it. Because we don't want it to be a required entry.

azd provision --resource-group ${rg}

How to support subscription or rg level deployments

Right now our templates default to sub level. Users may not have permissions to deploy to a sub, even if the resource group already exists.

I do not think it is possible to switch a bicep file targetScope at runtime. Asking the bicep team.

We may need to create two bicep files, one for sub and one for rg. But that should be the last resort.

[colbylwilliams]

Thanks for weighing in @jongio -- that all makes sense

Bicep defaults to targetScope='resourceGroup', so I'm curious why we don't. Instead of checking if the main bicep template isn't a subscription-level deployment, should we check if it is a subscription-level deployment?

I see a majority of the starter templates utilizing the existing pattern of having a top-level template that deploys at the subscription scope to create the resource group, then deploys the resources into that group -- which makes sense.

But instead of assuming this is always the case, should we check if the top-level template specifies targetScope='subscription' and if so continue with the sub-scoped deployment, otherwise assume a resource group is required?

That allows me to "azd-ify" my resource group templates without assuming I also have to always create the resource group.

Thoughts?

[jongio]

Interesting, we could inspect the bicep entry point, determine the scope and then make a decision there so it is flexible.

[markweitzel]

Adding some color to this thread... I was working with one of our MVPs, who indicated his customers use the following pattern for resource groups: rg-${location}-${env}-${appName}-01. IIRC, this is following the guidelines of the Cloud Adoption Framework.

I feel like this should be part of our naming epic, Azure/azure-dev-pr#947.

[colbylwilliams]

@markweitzel I think this is related to Azure/azure-dev-pr#947 but its not entirely about controlling the name, its also about scoping the deployment to a resource group instead of a subscription.

I've done some experimentation and there are other commands in azd that assume subscription scoped deployments, for example: azd evn refresh

The scenario I'm trying to get to work right now is using azd with Azure Deployment Environments where the developer (or service principal provided by ADE) will only have access to a resource group, not the whole subscription. So if azd did support resource group scoped templates/deployments, it would provide a great way for enterprises to jumpstart the creation of their "catalog" of blessed environments.

[savannahostrowski]

cc: @danieljurek I heard you have details to add 😸

[jongio]

@colbylwilliams and @danieljurek

Today, azd doesn't support the resourceGroup scope, but in the mean-time, can you see if this will work for you?

Have a look at the commit here: https://github.com/jongio/azdexistingresourcegroup/commit/753441812489713f4741bdb0115d92e4d0937be4

It still does a sub level deployment, but it allows you to use an existing resource group.

@rajeshkamal5050 and @savannahostrowski - Let's keep this item open to track supporting deploying to resourceGroup scope level. My comment above is a workaround.

[mattchenderson]

(Just sharing my experience with this in case it's helpful to anyone)

I've tried to work around this along the lines of Jon's suggestion, using multiple bicep files as different entry points. This allows me to take CI/CD at least off the azd path so that it can use the lower scope, while I still get the benefits of azd as-is for ad hoc test environments, etc. I'm very much trying to avoid giving whole subscription write access to GitHub Actions, and I accept that this means the resource group has to already exist for the target environment. I'm generally fine with this pattern, especially since the bootstrapping of the repo also requires me to have set up a user-assigned identity with a FIC and its role assignment so that the repo can do an identity-based deployment. Therefore, I'm already in the land of needing to pre-create resources. I'm fully ignoring azd pipeline config because of the scope concern.

So what I've done for some of my projects is have GHA kick off a group-scope resources.bicep directly through azure/arm-deploy@v1. main.bicep remains a subscription scope, but it simply does the group creation and kicks off a group-scoped sub-deployment using resources.bicep. The repo gets set up as part of a bootstrapping action that is scripted.

I wanted to ensure consistency in the setup of the repo's environment (get the tags right, etc.), so I did split the group creation into its own module that could be invoked during the repo setupas well. This is why I introduced additional subscription-scoped modules which just reference the group through resource ... existing and then set the scope property on later components. This could be cleaned up to use explicit resource functions I think, but I didn't find success with that at first and just kept going with these extra modules. And maybe trying to share that resourceGroup source of truth is overkill - within the script, it's easy enough to do the same steps with the Az CLI.

So I ended up with, beyond the usual group-scoped resource modules in the app folder:	Module	Scope	Action	Entry point notes
main	subscription	Calls app/main_resourceGroup and app/resources_subscriptionScope (passing along group name as a param)	Entry point for azd commands run locally with caller's permissions
resources	resourceGroup	Calls usual resource modules in app folder	Entry point for GHA via azure/arm-deploy@v1 with group-scoped permissions only
app/main_resourceGroup	subscription	Creates the resource group
app/resources_subscriptionScope (This could probably be collapsed into main)	subscription	References group and calls resources in that group's scope
repo/gha_setup	subscription	Calls app/main_resourceGroup and repo/gha_setup_identity	Entry point for repo bootstrapping run using az subscription deployment create with caller's permissions. To be run alongside the gh CLI to get the variables in place.
repo/gha_setup_identity (This could probably be collapsed into repo/gha_setup)	subscription	References the group and creates the managed identity, the FIC, and the role assignment scoped to the group.

It's a bit messy, but it's worked for my purposes. I have some issues where azd down isn't working fully correctly, but I think those are probably solvable. I also find it awkward to have to flow parameters through the extra hops, but I think that's just how it is.

[ellismg]

@mattchenderson As a note, in #2083 I am exploring what it would mean for azd to target deploying to a resource group directly (i.e. the root bicep you deploy would be targeting subscription scope). This might be interesting to you.

[OrionSehn]

Just wanted to bump this, and make it known that, at least I, and I'm sure others would very much appreciate this feature. I may try and give it an effort myself in the next month or two if nobody can get to it before me.

[savannahostrowski]

We recently added resource group scoped deployments! Will be announced on Wednesday with our monthly release.

You can give it a shot by running to enable it as an alpha feature.

azd config set alpha.resourceGroupDeployments on

When running azd provision or azd up on a repository with resource group scope infrastructure as code, you'll now be prompted to create a new resource group or select an existing resource group by name to provision to.

[rajeshkamal5050]

Marking this as complete. Should be part of 7/12 release.