Create subscriptions programmatically using Azure Resource Manager

[IrfanAnsari]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Create subscriptions programmatically using Azure Resource Manager. Here are the details about the API being released from Azure https://docs.microsoft.com/en-gb/azure/azure-resource-manager/programmatically-create-subscription

New or Affected Resource(s)

Subscriptions

• azurerm_XXXXX

Potential Terraform Configuration

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key.

References

• 0000

[Lachlan-White]

Hey team, what has blocked the progress of this one?

[tombuildsstuff]

@a138076 whilst we (or someone else) may be able to build support for this using the SDK - unfortunately we've current got no means of testing this (since we currently don't have an Enterprise Account, which is a requirement here).

@grayzu has been trying to determine if we can get an EA account (and a such I'm going to assign this to him) - however that's why this is blocked at this time.

[Lachlan-White]

@tombuildsstuff / @grayzu

I have an EA Account and happy to help if I can

[JGud]

I am also interested in this capability. Do you have any reccomendations for how to do this while we wait for the functionallity? Currently this article seems to describe a possible workaround? https://thegrayzone.co.uk/blog/2017/03/external-terraform-provider-powershell/

[mingfang]

This is high priority feature for us. How can we move this along?

[av8rgeek]

I have access to an Enterprise Account and am trying to solve this issue of creating subscriptions right now. I am happy to facilitate any testing that is needed. Please feel free to reach out to me directly.

[tombuildsstuff]

👋

To give an update here: after having spent some time investigating this, whilst there's now an SDK available for this (and there's an SDK for Rename/Cancel coming, but I can't find the link right now) - I don't believe it's possible to implement this at this point in time.

At this point in time the only API available for this allows creating a maximum of 50 subscriptions (total) and only for Enrollment Accounts. As we run acceptance tests for all resources in the Provider, we need to be able to create and destroy Subscriptions dynamically every test (which we run every night), thus we'd quickly hit our limit (and be unable to provision any more). In addition for customers, I'm not necessarily sure how useful it would be to only create 50 subscriptions, ever - since you can't also create Enrollment Accounts through the API.

All that to say - after chatting with Microsoft it sounds like they're working to remove that limitation - but there's not a (public) timeframe for that. As the SDK is available for this it should be fairly easy to add support for this once the API limitation's been removed - but since this is currently blocked on a fix in the Azure API I'm going to assign this to the Blocked milestone for the moment - but once the limitation's been removed we'll circle around and pick this up.

Thanks!

[wtejasukmana]

Did anyone get feedback on whether the limitation on Azure EA account subscription creation have been lifted by MS?

[chrheg]

Looks like there has been some changes now: There's a limit of 200 subscriptions per enrollment account. After that, more subscriptions for the account can only be created in the Azure portal. If you want to create more subscriptions through the API, create another enrollment account. https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/programmatically-create-subscription?tabs=rest

[wtejasukmana]

progress I guess, although this won't resolve the issue at hand :(

[adarobin]

Could this be added without an acceptance test for this resource? We only create 1 or 2 subscriptions a month on average a limit of 200 would last us several years.

[taylorturner]

Very interested in this feature! Thank you to those who are working on pushing it forward.

[tombuildsstuff]

@chrheg thanks for the heads up - 200 is better than 50, but it's still fundamentally the same problem unfortunately

@adarobin unfortunately not - in practice the acceptance tests have caught a ton of issues for us when they're run both nightly and when we upgrade the Azure SDK (since the SDK is auto-generated and contains no tests in itself) - as such it's a requirement for shipping any resource to be able to confirm it works/continue shipping it.

Whilst I appreciate it's frustrating this feature is blocked - it looks like progress is being made in the Azure API - I'll reach out to Microsoft on our end to see if there's any public timeframe for this too.

Thanks!

[timja]

@tombuildsstuff what's the issue with only being able to create 200? can't you cancel them?

I would assume you aren't creating 200+ in parallel?

[tombuildsstuff]

@timja you can create 200 subscriptions through the API, periodg (which still count even if they're cancelled) - rather than concurrently/a quota unfortunately. So running say a couple of tests nightly plus during SDK upgrades (and development) means we'd be unable to run the tests/provision any more subscriptions after a few months - which is the issue here

Once it's possible to provision these on demand this should be fine - but for the moment unfortunately the limit of creating 200 subscriptions (ever) means this isn't something we can ship until that's fixed unfortunately.

[timja]

oh, I didn't realise, completely understand with that limitation then.

[TheKangaroo]

Okay, so you have to use a new service principal every 200 subscription creations, or am I missing something here? Even though this is kind of annoying, it would be a huge improvement if we could create subscriptions within a terraform deployment.

[timja]

Okay, so you have to use a new service principal every 200 subscription creations, or am I missing something here? Even though this is kind of annoying, it would be a huge improvement if we could create subscriptions within a terraform deployment.

No you need a new enrolment account AFAIK

[tombuildsstuff]

No you need a new enrolment account AFAIK

Correct - and my understanding (although I might be wrong) is that's paperwork to create a new enrollment account rather than an API, as such I don't think we can create those via an API either?

[timja]

No you need a new enrolment account AFAIK

Correct - and my understanding (although I might be wrong) is that's paperwork to create a new enrollment account rather than an API, as such I don't think we can create those via an API either?

Not that I'm aware of, I think they can just be created in the EA portal, I have my own enrolment account created for me by my company.

But I don't have access to create new ones so can't be sure

[logachev]

Would be great to have this feature in Terraform!

Looking at MSDN docs and it looks like current limit was increased to 500.

[shelwig]

@tombuildsstuff I escalated this through our engineering team and they have raised the limit to 500. Could we get this implemented as soon as possible? Please reach out to me directly on Github and we can connect offline.

[pearcec]

Here is a link to the APIs for subscriptions. The upstream Azure/azure-sdk-for-go module will need updated for 2019-03-01-preview. The biggest ongoing issue is when you cancel in an EA (not sure if this is true for Pay-as-you-go or CSP), azure holds the subscription for 90 days. See information here. I don't know if this can be changed. I assume we would want to do both EA and Pay-as-you-go (and CSP?).

[TheKangaroo]

We are Azure EA customers and I can confirm, that azure holds our subscriptions for 90 days after deletion. Actually this is the desired behavior from our point of view. I think from terraform view we could treat a subscriptions on hold the same way, as a deleted subscription (e.g. as if it does not exist). Or am I missing something? Btw. we're really looking forward for azurerm provider to support subscription creation :)

[pearcec]

We are Azure EA customers and I can confirm, that azure holds our subscriptions for 90 days after deletion. Actually this is the desired behavior from our point of view. I think from terraform view we could treat a subscriptions on hold the same way, as a deleted subscription (e.g. as if it does not exist). Or am I missing something? Btw. we're really looking forward for azurerm provider to support subscription creation :)

Yes we still want this behavior. But if you are running acceptance tests it becomes and issue when you add and remove it 50 times a night and then you exhaust your supply in 10 days.

[petems]

If people do want to implement this in their own customised provider, dnaeon/go-vcr might be the way you'd test this, otherwise you're going to exhuast your pool of account creation pretty quickly...

[TheKangaroo]

Ok, I see, that's in fact a problem I didn't think about 😇

[tombuildsstuff]

:wave:

To give an update from our side here: based on the quota's that exist today (essentially 500 subscriptions/90 days) - unfortunately this quota still isn't enough for us to be able to ship and test this - but we're getting close.

To give some math here: allowing for 50 subscriptions / test runs to get this built & merged - we've then got (say) 5 tests a night, plus additional times we run the tests (e.g. SDK upgrades), which is over 500 for a 90 day period, but not by much.

Based on the back-of-the-envelope math above, we're getting close to where this is possible - once the limit reaches (say) 1000/2000 we should be able to take another look at this. From our side there's additional tests we'd need to add over time (e.g. permissions/configuration at a subscription level), so a fixed limit this low isn't ideal - but this should allow us to at least ship this resource.

As mentioned above we're working with Microsoft to make this happen - we're also running down the internal testing related things to make this happen - so we're hoping that by the time these limits get a little higher we should be in a position to take another look/ship this :+1:

Thanks!

---

@petems unfortunately the nature of the Azure API's means the VCD testing wouldn't be sufficient here

[TheKangaroo]

Okay, quick question, I thought the limit of 500 subscriptions is "subscription creation through API" as in "one enrollment account can call the create subscription endpoint 500 times" and not "one enrollment account can have up to 500 subscriptions created via API in parallel". Is this assumption wrong? 🤔

[wilbug1git1]

Hi Tom. I have been emailing with Katie on this issue to help you get a quota increase for you Azure sub. Can you reach out to Katie and reach me on that thread? thanks!

[petr-stupka]

Hi @tombuildsstuff, @wilbug1git1, do you think you guys will be able to make progress on it together? Most of today's customers have EA account so it's actually a nice feature. Thanks for the great job you guys doing!

[joakimhellum]

We are Azure EA customers and I can confirm, that azure holds our subscriptions for 90 days after deletion. Actually this is the desired behavior from our point of view. I think from terraform view we could treat a subscriptions on hold the same way, as a deleted subscription (e.g. as if it does not exist). Or am I missing something?

We have been testing this internally since v42.3.0 release of Azure SDK for Go.

Example resource for creating subscription in enrollment account: https://github.com/innovationnorway/terraform-provider-azurerm/commit/cd40af6f23a59726628ec9147f3f1d947e0c53f6

Our only concern so far, besides from the subscription limit already mentioned, is that creating and deleting (cancelling) subscriptions requires the RBAC Owner role both on enrollment account (ref. https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/grant-access-to-create-subscription?tabs=rest%2Crest-2) and subscription (ref. https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/cancel-azure-subscription#who-can-cancel-a-subscription). However this might not be a concern for most 😄

[J-i-K]

Looks like the limit was raised to 2000 subscriptions per enrollment account, is that sufficient and/or has there been any possibility to progress this further?

[sangeethaguhan]

Can't wait for this functionality to be implemented asap!

[jameswestall]

Can anyone advise at this point in time does the current update only support EA subscriptions?

[Marcus-James-Adams]

From what I understand the 2000 subscription limit is now in place, are there any other blockers that are preventing this moving forwards and being included in automatic testing?

[gurrish]

Does anyone know when this feature will be included?

[J-i-K]

Could someone confirm if I've now understood this correctly, that the MS APIs that support this* are still in preview? If so, I take it that is the blocker? And again if so, would someone happen to have any insights into when this might be released? I understood that there are ppl working in collaboration with MS around this..?

*Managing EA Subscriptions through APIs

[rrey]

Hi all, could we have an update on this issue. Things looks to be in place to have something integrated in the official provider. People at innovationnorway looks to have the code and the tests available and the hard limit have been raise to 2000.

Am I wrong or do we just need the PR and the review process ?

[tombuildsstuff]

@rrey alas there's some upcoming changes coming on the Azure side that'll mean the existing API is unsuitable (it's being superseded) - we're working with the teams at Microsoft to unblock that - but we'll post more information when we have it.

[amochtar]

@tombuildsstuff Could you elaborate what APIs will change? We're doing something similar as @joakimhellum describes (but then with an MCA based account, rather than a EA account), so I am curious if that will impact us.

[jackofallops]

@amochtar - I'll be working on this soon, and we're talking through the changes with MSFT at the moment, so it's not something we have pinned down right now. I'll link the work here when I start.

[matt-FFFFFF]

The schema for the new API is published: https://github.com/Azure/azure-resource-manager-schemas/blob/master/schemas/2020-09-01/Microsoft.Subscription.json

'aliases' is the resource type to create new subscriptions

[jackofallops]

The schema for the new API is published: https://github.com/Azure/azure-resource-manager-schemas/blob/master/schemas/2020-09-01/Microsoft.Subscription.json

'aliases' is the resource type to create new subscriptions

Thanks @matt-FFFFFF - I have that and chatting though implementation options with our MSFT Azure PM, slight delays due to timezone differences. Hoping to get started soon.

[rogerm-chen]

for azurerm_resource_group_template_deployment, resource will automatically attempt to delete resources deployed by the ARM Template when it is deleted but not the same case for azurerm_subscription_template_deployment, can this be improved?

[tombuildsstuff]

@theasphaltworld please open another issue to track that - since that request is unrelated to this issue

[rb-cloud-guru]

Hi I can’t tell from the above comments: are there any potential chance this feature will be available for EA customers? Thanks

[jackofallops]

Hi I can’t tell from the above comments: are there any potential chance this feature will be available for EA customers? Thanks

We have ongoing conversation with Microsoft on this, as soon as we have a clear path for what is and isn't supported we'll be creating this / these resources (we really want this feature too!)

[Tailzip]

Hi, any updates on this ? 🙂

[laingsc]

Anyone looking for a way to do this while waiting, you can use this provider: https://registry.terraform.io/providers/innovationnorway/azure-preview/latest/docs/resources/azurepreview_subscription