jason-johnson / azure-pipelines-tasks-terraform

Azure Pipelines extension for Terraform
MIT License
121 stars 52 forks source link

Working with multiple subscriptions #355

Closed slaterx closed 10 months ago

slaterx commented 1 year ago

Hi Team,

I'm trying to understand how to use this to implement terraform code with multiple subscriptions.

In plain terraform, we would have the following to define the aliases of our subscriptions, say Azure for argument's sake:


# Configuration for our "main" subscription
provider "azurerm" {
  tenant_id       = var.TENANT_ID_MAIN
  subscription_id = var.SUBSCRIPTION_ID_MAIN
  client_id       = var.SERVICE_PRINCIPAL_ID_MAIN
  client_secret   = var.SERVICE_PRINCIPAL_SECRET_MAIN
  features {}
}

# Configuration for the "secondary" subscription
provider "azurerm" {
  alias           = "secondary"
  tenant_id       = var.TENANT_ID_SECONDARY
  subscription_id = var.SUBSCRIPTION_ID_SECONDARY
  client_id       = var.SERVICE_PRINCIPAL_ID_SECONDARY
  client_secret   = var.SERVICE_PRINCIPAL_SECRET_SECONDARY
  features {}
}

However, since TerraformTasks expects a parameter called backendServiceArm, where we would have defined the service connection to the cloud provider, this is how we define them inside terraform:

provider "azurerm" {
  features {
  }
}

Hence, it's not clear to me how we define the already implemented secondary service connections in Azure DevOps for multiple subscriptions. Could you shed some light on how to implement this?

jason-johnson commented 1 year ago

hi, if the service principal can access both subscriptions I think it should be enough to just specify the subscript_id and use backendServiceArm as normal. Have you tried that?

slaterx commented 1 year ago

I haven't. I'll try and get back to you.

In a scenario where that's not possible, how can we access multiple subscriptions using service connections only?

jason-johnson commented 1 year ago

Well, this extension is just setting up and making the calls to the terraform CLI, so how would you do it if you were running the commands manually in the agent?

anthonysomerset commented 11 months ago

just want to state i have this issue too

if i setup with the main subscription for my remote state backend and my tf files have this

provider "azurerm" {
  subscription_id = "xxxxxxxx-xxxx-xxxx-xxxx-5af85f7fb97c"
  features {}
}

provider "azurerm" {
  alias = "it-shared"
  subscription_id = "yyyyyyyy-yyyy-yyyy-yyyy-c00645d69e56"
  features {}
}

provider "azurerm" {
  alias = "ct-identity"
  subscription_id = "zzzzzzzz-zzzz-zzzz-zzzz-ecfc60d3ba7e"
  features {}
}

when the terraform plan attempts to run i get multiple errors like this:

Error: Error ensuring Resource Providers are registered.
│ 
│ Terraform automatically attempts to register the Resource Providers it supports to
│ ensure it's able to provision resources.
│ 
│ If you don't have permission to register Resource Providers you may wish to use the
│ "skip_provider_registration" flag in the Provider block to disable this functionality.
│ 
│ Please note that if you opt out of Resource Provider Registration and Terraform tries
│ to provision a resource from a Resource Provider which is unregistered, then the errors
│ may appear misleading - for example:
│ 
│ > API version 2019-XX-XX was not found for Microsoft.Foo
│ 
│ Could indicate either that the Resource Provider "Microsoft.Foo" requires registration,
│ but this could also indicate that this Azure Region doesn't support this API version.
│ 
│ More information on the "skip_provider_registration" flag can be found here:
│ https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs#skip_provider_registration
│ 
│ Original Error: populating Resource Provider cache: listing Resource Providers: loading the initial page: providers.ProvidersClient#List: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '<CLIENTGUID>' with object id '<CLIENTGUID>' does not have authorization to perform action 'Microsoft.Resources/subscriptions/providers/read' over scope '/subscriptions/yyyyyyyy-yyyy-yyyy-yyyy-c00645d69e56' or the scope is invalid. If access was recently granted, please refresh your credentials."
│ 
│   with module.core-connectivity.provider["registry.terraform.io/hashicorp/azurerm"].it-shared,
│   on modules/connectivity/main.tf line 21, in provider "azurerm":
│   21: provider "azurerm" {
│ 

once i re-read this as i pasted this here i realised my SPN needed permissions to the additional subs - applied the appropriate permissions to the subs manually and it succeeded just fine

what might be good is some documentation notes for this use case

jason-johnson commented 10 months ago

just want to state i have this issue too

... what might be good is some documentation notes for this use case

Hi @anthonysomerset , thanks for the update. One thing to keep in mind: this extension is just calling terraform from an Azure Devops Pipeline. The things discussed in this thread are actually terraform subjects. And the error you found is discussed e.g. here. We don't want to duplicate or even try to link to any terraform documentation due to the effort involved in keeping up with changes they make.

jason-johnson commented 10 months ago

Hi @slaterx, did you figure it out?

jason-johnson commented 10 months ago

Looks like this is not an issue with this extension. Closing.