Azure / terraform-azurerm-caf-enterprise-scale

Azure landing zones Terraform module
https://aka.ms/alz/tf
MIT License
856 stars 565 forks source link

Feature Request: Assigning Policies to Subscriptions (roadmap query) #337

Closed rfk-nc closed 1 year ago

rfk-nc commented 2 years ago

Community Note

Description

Hi,

We are using the caf-es module for a large UK public sector customer as part of a standardised landing zone deployment that they will use for internal and "internal customer" projects.

One requirement we need to meet is assigning policies to subscriptions (in addition to management groups), and can see that this is work in progress in the current module.

Is there any indication when this functionality will be completed, or any good suggestions on applying the existing policy definition files to subscriptions?

Thanks! Richard

krowlandson commented 2 years ago

Thank you for this question @rfk-nc

Currently Azure landing zones only implement Policy Assignments at the Management Group scope. This was by design and there is currently nothing on the roadmap to change this.

It's great that you've discovered this in the code for this module. As you have rightly identified, there are some thoughts around whether we could implement this as a feature in the future. The thinking behind this was whether we could use the module to effectively apply an "application specific" archetype to a Subscription, allowing the same templating approach to be used at the Subscription scope as we use for Management Groups.

We don't currently have plans to implement this, but I would love to get your feedback on this capability to better understand how you would like to use this, and what "good" might look like.

rfk-nc commented 2 years ago

Hi Kevin,

Many thanks for getting back to me, appreciate the response.

The reason we are interested in this functionality is due to the distributed nature of our customer's environment, with a central IT team delegating many responsibilities to a departmental level. Some departments will only have one or two applications within a single subscription, but others are much larger and will have multiple subscriptions across multiple management groups.

In fact, we envision the Landing Zone MG structure to look something like this, eventually with 50+ subscriptions in place:

-Org Root

The issue is that, as mentioned, some applications will be comprised of five or six subscriptions, so while we can locate these within their own MG, the subscriptions themselves may still have divergent policy requirements, as some will contain e.g. public vs private connectivity, or different resource types.

We are still just about within the limit of six MG levels, so if we have to use additional layers of those, that could be a possible solution for us, but I thought it worth checking on applying policies directly to subscriptions as well as we'd seen the initial work you've done on this :)

Would be happy to hear any other recommendations you might have as this could help us steer the conversation with the customer.

(Edit: and yes, we realise this is a fairly specific use case / edge case!)

As an aside, we have done quite a lot of other customisation on top of the es-caf module that I'm sure you would be interested in seeing too (e.g. defining our Management Groups fully within the json archetype files and loading these as custom landing zones, so we have separation between logic and config, but can also dynamically add new MGs by simply dropping a new archetype file into the repo).

We'd be happy to share this with you, when we have it polished up a bit better!

Thanks, Richard

matt-FFFFFF commented 2 years ago

Thanks for your detailed use case information @rfk-nc

We will keep this issue open to track the request and prioritize accordingly.

MarcelHeek commented 2 years ago

Also interested, as we could drop additional layer of MG's if this were possible

krowlandson commented 2 years ago

Also interested, as we could drop additional layer of MG's if this were possible

@MarcelHeek, I assume you're thinking in the context of the Subscriptions under platform, as this is a common ask?

If so, we place these Subscriptions in dedicated Management Groups rather than using Subscription scoped assignments as this prevents a Subscription Owner from removing the Policy Assignment. This is important from a governance enforcement perspective.

MarcelHeek commented 2 years ago

@krowlandson No, same as original requester I am referring to the Landing Zones MG structure. image In the example above we use a PRD and non-PRD child MG for applications to differentiate between subscriptions for production and non production. Just to be able to assign separate Archetypes definitions. If we were able to assign these Archetypes to the subs directly we would no longer need the PRD / non-PRD MG layer

krowlandson commented 2 years ago

Thank you for the additional details @MarcelHeek... the same issue I mention above still exists, but if you're not concerned about the possibility of a Subscription Owner removing these assignments (i.e. they are not crucial to governance) then this I agree this might be useful.

We will discuss this internally in the broader context of the Azure landing zones design recommendations to see whether there are updates we can make to the guidance which would help us to prioritize this feature.

Also, thank you to @rfk-nc for the additional context. This will be useful for our discussions.

rfk-nc commented 2 years ago

Hi @krowlandson @MarcelHeek The point about subscription owners removing the assignments is a really good one I think, and we have also been able to persuade our customer that applying all policies at MG level rather than a mix of MG and subscription is simply more transparent.

We have therefore settled on a management group structure along the following lines: image

.. and will add additional MGs at the lowest level of the hierarchy on a case by case basis if required.

jtracey93 commented 2 years ago

Trigger ADO Sync

krowlandson commented 2 years ago

Trigger ADO Sync

Eslam10 commented 2 years ago

Hello, I have the same use case. In our case we have some custom child MGs under LandingZones MG, some of the subscriptions require additional policies (subscription specific), for instance some subscriptions are used for a certain application on which we need to limit the resource types that will be created on these subscriptions using Azure policy. Such policy can not be applied on the MG level, and it is different for each subscription.

jtracey93 commented 2 years ago

Hey @Eslam10, I would encourage you to consider if these shouldn't be additional management group as per our tailoring ALZ guidance here: https://learn.microsoft.com/azure/cloud-adoption-framework/ready/landing-zone/tailoring-alz

As if the policies need to be enforced we should place them correctly on MGs so sub owners cannot remove them.

krowlandson commented 2 years ago

Thank you for the additional support on this item @Eslam10 👍🏻

The voice of our community is important and helps us to prioritize these items, however from a technical perspective this will actually be challenging to implement due to the provider authentication model which links it to a single Subscription for resource creation.

We are evaluating the AzApi provider to see whether we could integrate this to overcome the scoping challenge but there's still the bigger question around whether this is something we would implement in the module as this approach technically falls outside of the ALZ recommendations.

Please also note @jtracey93's point above around the implications in regard to enforcing governance.

Eslam10 commented 2 years ago

@krowlandson, Thanks for the feedback. I did a simple test maybe it will help in the case evaluation.

I tested the azurerm_subscription_policy_assignment resource with only one provider authentication to one subscription and applied the code to two different subscriptions.

Since this resource has an argument subscription_id, it successfully applied the policy to the two different subscriptions, without using provider alias in the TF resource.
For sure, the used SPN should have access to all the subscriptions that this resource will be applied to, which is in my case I applied the permission on the MG level.

krowlandson commented 2 years ago

Thank you @Eslam10 ... will definitely check this out as I hadn't appreciated this resource was an exception to the "normal" rule of Subscription scoped resources 😄

However, we still need to validate the business benefit of adding this to the module, and how it aligns with the broader ALZ guidance so cannot make any promises to add this just yet.

Of course, community contributions are also welcome 😉

matt-FFFFFF commented 1 year ago

Hi, after discussion we cannot add this feature to this module. However we do have a sibling module that we may be more suitable for this ask.

Please check out https://github.com/Azure/lz-vending for more info

matt-FFFFFF commented 1 year ago

See Azure/terraform-azurerm-lz-vending#117