Unable to specify subscription ID for backend in terraform init when multiple subscriptions are available

[JayDoubleu]

Currently, when running terraform init with a Service Connection that has access to more than one subscription, the command fails to specify the correct subscription ID for the backend. The -backend-config=subscription_id= option is set, causing the command to fail with an error.

Attempts to add the -backend-config=subscription_id=xxxx option to commandOptions in the pipeline also fail, as it is prepended to the command instead of appended, and the empty -backend-config=subscription_id= overrides the manually specified subscription ID.

This issue could be potentially fixed if commandOptions were appended instead of prepended to the terraform binary, allowing the subscription ID to be properly specified.

[mericstam]

Thanks for reporting. We will put this issue on our internal task board. no ETA due to low bandwidth at the moment

[DenisBalan]

Please implement this feature! For now we should use non-official task

task: TerraformCLI

https://github.com/jason-johnson/azure-pipelines-tasks-terraform/blob/main/overview.md

Which has this ability with

backendAzureRmSubscriptionId: 'my-backend-subscription-id'

[JoakimBG]

This is a blocker for my projects as well. We have several environments that have state stored in a management subscription and don't want to have that available in each subscription.

[mericstam]

As a workaround you can have two service connections in your Azure devops project configured. Like one for management subscription where you store state and one for the actutal resources. ex. backendSPN and resourceSPN. for the terraform init you use the backendSPN and for the resoucres you use the resourceSPN (the plan and apply stuff). Are you unable to have two serviceconnections? This is how I do it. I have state in one Azure Subscription and the actual resources in another.

[JoakimBG]

I have seen your reply (https://github.com/microsoft/azure-pipelines-terraform/issues/67#issuecomment-1281219605).

That might work, not tested yet. We don't want to share access to different environment states between environments / departments / solutions. Also still there is risk of altering state for other deploys as you have access to them as well instead of only the SA for that environment.

So that suggestion you have is not complete for security point of view. There might be secrets stored that should not be available to others...

Running latest TF, TerraformTaskV4@4 extention

[mericstam]

I guess you could limit the backendSPN to just one resource, where that specific state files resides, but that will get to be complicated to maintain if you have many solutions that have their own state. I might totally missunderstand what you are trying to accomplish and I did not read the code for the suggested non-official extension but changing backendAzureRmSubscriptionId will that be more secure then having two service connections? the service connection still have to have access to the specified subscriptions resources?

[JayDoubleu]

For the record, having single SP to manage more than single subscription is a horrible security practice so I understand why this isn't prioritised. Perfect example of "just because I can, doesn't mean I should"

[JoakimBG]

Hi @JayDoubleu , Think there is an misunderstanding roaming in your reply... I'm not talking about single SP for entire environment. Quite the opposite... I want to have dedicated SP for each environment that ONLY have access to its own SA (in other subscription) for storing tfstate. This way you enhance security for state as well as storing it outside your deployed environment. This facilitates several layers of security as well.

[rruenroeng]

I am running into this same problem for my infrastructure deployment pipeline. The TerraformTaskV4@4 task does not allow me to specify which subscription I'd like to use for my backend and I've been running into several issue as a result.

[theo-albers]

I'm having a problem as well. I have two service connections:

• 1 background service connection, pointing to subscription A, tenant A
  • https://dev.azure.com/foo/bar/_settings/adminservices?resourceId=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
  • subscription aaaaaaaa-aaaa-aaaa-aaaa-bbbbbbbbbbbb
  • variable TerraformBackendServiceConnection
• 1 deployment service connection, pointing to subscription B, tenant A
  • https://dev.azure.com/foo/bar/_settings/adminservices?resourceId=ffffffff-ffff-ffff-ffff-ffffffffffff
  • subscription ffffffff-ffff-ffff-ffff-bbbbbbbbbbbb
  • variable ResourceServiceConnection

When I enable logging TF_LOG=TRACE in my pipeline, download the logs, I can see that the task resolves the wrong service connection for environmentServiceNameAzureRM. Here is a snippet of the plan prepare pipeline log:

2023-09-28T08:21:25.4591150Z ##[section]Starting: Prepare job Plan
2023-09-28T08:21:25.5216160Z Variables:
2023-09-28T08:21:25.5216160Z   ResourceServiceConnection: $[ variablegroups.test-env.ResourceServiceConnection ]
2023-09-28T08:21:25.5216160Z   TerraformBackendResourceGroupName: $[ variablegroups.test-env.TerraformBackendResourceGroupName ]
2023-09-28T08:21:25.5216160Z   TerraformBackendServiceConnection: $[ variablegroups.test-env.TerraformBackendServiceConnection ]
2023-09-28T08:21:25.5216160Z   TerraformBackendStorageAccountName: $[ variablegroups.test-env.TerraformBackendStorageAccountName ]

And the plan pipeline log:

2023-09-28T08:21:46.4775808Z ##[debug]/opt/hostedtoolcache/terraform/1.5.7/x64/terraform arg: init
2023-09-28T08:21:46.4778692Z ##[debug]backendServiceArm=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
2023-09-28T08:21:46.4782223Z ##[debug]backendServiceArm=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
2023-09-28T08:21:46.4783813Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth scheme = ServicePrincipal
2023-09-28T08:21:46.4784477Z ##[debug]Setting up backend for authorization scheme: serviceprincipal.
2023-09-28T08:21:46.4785355Z ##[debug]backendAzureRmStorageAccountName=icraterraforms

....
2023-09-28T08:21:55.8655848Z ##[debug]/opt/hostedtoolcache/terraform/1.5.7/x64/terraform arg: plan
2023-09-28T08:21:55.8656309Z ##[debug]/opt/hostedtoolcache/terraform/1.5.7/x64/terraform arg: -var-file=environment.tfvars -out=infrastructure.tfplan -detailed-exitcode
2023-09-28T08:21:55.8656788Z ##[debug]environmentServiceNameAzureRM=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
2023-09-28T08:21:55.8657144Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth scheme = ServicePrincipal
2023-09-28T08:21:55.8657489Z ##[debug]Setting up provider for authorization scheme: serviceprincipal.
2023-09-28T08:21:55.8657869Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa data subscriptionid = aaaaaaaa-aaaa-aaaa-aaaa-bbbbbbbbbbbb
2023-09-28T08:21:55.8658258Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa data subscriptionid = aaaaaaaa-aaaa-aaaa-aaaa-bbbbbbbbbbbb
2023-09-28T08:21:55.8658652Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth param tenantid = xxx
2023-09-28T08:21:55.8659127Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth param serviceprincipalid = ***
2023-09-28T08:21:55.8659612Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth param serviceprincipalkey = ***
2023-09-28T08:21:55.8659981Z ##[debug]Finished up provider for authorization scheme: serviceprincipal.

I expected to see: environmentServiceNameAzureRM=ffffffff-ffff-ffff-ffff-ffffffffffff

This is a snippet of the pipeline:

...
stages:
- stage: DeployTest
  variables:
  - group: test-env
...
  jobs:
  - job: Plan
    steps:
    - task: TerraformTaskV4@4
      displayName: Terraform init
      inputs:
        command: init
      workingDirectory: $(System.DefaultWorkingDirectory)/infrastructure/applications
      backendAzureRmResourceGroupName: $(TerraformBackendResourceGroupName)
      backendAzureRmStorageAccountName: $(TerraformBackendStorageAccountName)
      backendAzureRmContainerName: ${{ parameters.TerraformAppName }}-${{ parameters.TerraformEnvironment }}
      backendAzureRmKey: ${{ parameters.TerraformAppName }}-${{ parameters.TerraformEnvironment }}.tfstate
      backendServiceArm: $(TerraformBackendServiceConnection)
      condition: succeeded()

    - task: TerraformTaskV4@4
      displayName: Terraform plan
      name: PlanTask
      inputs:
        command: plan
        workingDirectory: $(System.DefaultWorkingDirectory)/infrastructure/applications
        commandOptions: -var-file=environment.tfvars -out=infrastructure.tfplan
        environmentServiceNameAzureRM: $(ResourceServiceConnection)
      condition: succeeded()

[theo-albers]

Could it be related to the fact that the backend principal resolution uses "backendServiceArm" in all calls, but the handleProvider uses "AzureRM" instead of trying "environmentServiceNameAzureRM"? See relevant snippets below:

export class TerraformCommandHandlerAzureRM extends BaseTerraformCommandHandler {

    private async setupBackend(backendServiceName: string) : Promise<void> {
        const authorizationScheme = this.mapAuthorizationScheme(tasks.getEndpointAuthorizationScheme(tasks.getInput("backendServiceArm", true), true));
        this.backendConfig.set('subscription_id', tasks.getEndpointDataParameter(backendServiceName, "subscriptionid", true));
        this.backendConfig.set('tenant_id', tasks.getEndpointAuthorizationParameter(backendServiceName, "tenantid", true));
        var servicePrincipalCredentials = this.getServicePrincipalCredentials(backendServiceName);
        ....
    }

    public async handleBackend(terraformToolRunner: ToolRunner): Promise<void> {
        ...
        let backendServiceName = tasks.getInput("backendServiceArm", true);
        await this.setupBackend(backendServiceName);
        ...
    }

    public async handleProvider(command: TerraformAuthorizationCommandInitializer) : Promise<void> {
       ...command.serviceProvidername --> "AzureRM"
       const authorizationScheme = this.mapAuthorizationScheme(tasks.getEndpointAuthorizationScheme(tasks.getInput("environmentServiceNameAzureRM", true), true));
       const subscriptionId = tasks.getEndpointDataParameter(command.serviceProvidername, "subscriptionid", true);
       process.env['ARM_SUBSCRIPTION_ID']  = tasks.getEndpointDataParameter(command.serviceProvidername, "subscriptionid", false);
       process.env['ARM_TENANT_ID'] = tasks.getEndpointAuthorizationParameter(command.serviceProvidername, "tenantid", false);
       var servicePrincipalCredentials = this.getServicePrincipalCredentials(command.serviceProvidername);
       process.env['ARM_CLIENT_ID'] = servicePrincipalCredentials.servicePrincipalId;
       process.env['ARM_CLIENT_SECRET'] = servicePrincipalCredentials.servicePrincipalKey;
       ...
    }

[theo-albers]

I'm having a problem as well. I have two service connections:

• 1 background service connection, pointing to subscription A, tenant A

  • https://dev.azure.com/foo/bar/_settings/adminservices?resourceId=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
  • subscription aaaaaaaa-aaaa-aaaa-aaaa-bbbbbbbbbbbb
  • variable TerraformBackendServiceConnection

• 1 deployment service connection, pointing to subscription B, tenant A

  • https://dev.azure.com/foo/bar/_settings/adminservices?resourceId=ffffffff-ffff-ffff-ffff-ffffffffffff
  • subscription ffffffff-ffff-ffff-ffff-bbbbbbbbbbbb
  • variable ResourceServiceConnection

When I enable logging TF_LOG=TRACE in my pipeline, download the logs, I can see that the task resolves the wrong service connection for environmentServiceNameAzureRM. Here is a snippet of the plan prepare pipeline log:

2023-09-28T08:21:25.4591150Z ##[section]Starting: Prepare job Plan
2023-09-28T08:21:25.5216160Z Variables:
2023-09-28T08:21:25.5216160Z   ResourceServiceConnection: $[ variablegroups.test-env.ResourceServiceConnection ]
2023-09-28T08:21:25.5216160Z   TerraformBackendResourceGroupName: $[ variablegroups.test-env.TerraformBackendResourceGroupName ]
2023-09-28T08:21:25.5216160Z   TerraformBackendServiceConnection: $[ variablegroups.test-env.TerraformBackendServiceConnection ]
2023-09-28T08:21:25.5216160Z   TerraformBackendStorageAccountName: $[ variablegroups.test-env.TerraformBackendStorageAccountName ]

And the plan pipeline log:

2023-09-28T08:21:46.4775808Z ##[debug]/opt/hostedtoolcache/terraform/1.5.7/x64/terraform arg: init
2023-09-28T08:21:46.4778692Z ##[debug]backendServiceArm=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
2023-09-28T08:21:46.4782223Z ##[debug]backendServiceArm=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
2023-09-28T08:21:46.4783813Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth scheme = ServicePrincipal
2023-09-28T08:21:46.4784477Z ##[debug]Setting up backend for authorization scheme: serviceprincipal.
2023-09-28T08:21:46.4785355Z ##[debug]backendAzureRmStorageAccountName=icraterraforms

....
2023-09-28T08:21:55.8655848Z ##[debug]/opt/hostedtoolcache/terraform/1.5.7/x64/terraform arg: plan
2023-09-28T08:21:55.8656309Z ##[debug]/opt/hostedtoolcache/terraform/1.5.7/x64/terraform arg: -var-file=environment.tfvars -out=infrastructure.tfplan -detailed-exitcode
2023-09-28T08:21:55.8656788Z ##[debug]environmentServiceNameAzureRM=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
2023-09-28T08:21:55.8657144Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth scheme = ServicePrincipal
2023-09-28T08:21:55.8657489Z ##[debug]Setting up provider for authorization scheme: serviceprincipal.
2023-09-28T08:21:55.8657869Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa data subscriptionid = aaaaaaaa-aaaa-aaaa-aaaa-bbbbbbbbbbbb
2023-09-28T08:21:55.8658258Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa data subscriptionid = aaaaaaaa-aaaa-aaaa-aaaa-bbbbbbbbbbbb
2023-09-28T08:21:55.8658652Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth param tenantid = xxx
2023-09-28T08:21:55.8659127Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth param serviceprincipalid = ***
2023-09-28T08:21:55.8659612Z ##[debug]aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa auth param serviceprincipalkey = ***
2023-09-28T08:21:55.8659981Z ##[debug]Finished up provider for authorization scheme: serviceprincipal.

I expected to see: environmentServiceNameAzureRM=ffffffff-ffff-ffff-ffff-ffffffffffff

This is a snippet of the pipeline:

...
stages:
- stage: DeployTest
  variables:
  - group: test-env
...
  jobs:
  - job: Plan
    steps:
    - task: TerraformTaskV4@4
      displayName: Terraform init
      inputs:
        command: init
      workingDirectory: $(System.DefaultWorkingDirectory)/infrastructure/applications
      backendAzureRmResourceGroupName: $(TerraformBackendResourceGroupName)
      backendAzureRmStorageAccountName: $(TerraformBackendStorageAccountName)
      backendAzureRmContainerName: ${{ parameters.TerraformAppName }}-${{ parameters.TerraformEnvironment }}
      backendAzureRmKey: ${{ parameters.TerraformAppName }}-${{ parameters.TerraformEnvironment }}.tfstate
      backendServiceArm: $(TerraformBackendServiceConnection)
      condition: succeeded()

    - task: TerraformTaskV4@4
      displayName: Terraform plan
      name: PlanTask
      inputs:
        command: plan
        workingDirectory: $(System.DefaultWorkingDirectory)/infrastructure/applications
        commandOptions: -var-file=environment.tfvars -out=infrastructure.tfplan
        environmentServiceNameAzureRM: $(ResourceServiceConnection)
      condition: succeeded()

I verified with another pipeline that all variables have been set correctly. I can rule out pipeline config issues. The limitation seems to be that environmentServiceNameAzureRM isn't used to pick up the tenant/subscription/service principal during a custom/plan/apply command.

[mericstam]

@theo-albers I can confirm this no longer work as expected with two service connections. I will take a look. this is error is unrelated the above thread which is about two subs but one service connection. I will create a separate Issue

[speak2beeb]

Yes! This would be helpful at my organization as well!

There is another workaround which I'm on the fence whether to use it or not. In any case, the workaround would simply be running the CLI command without the task like:

terraform init -backend-config=storage_account_name=MYACCT -backend-config=container_name=MYCONT -backend-config=key=MYKEY.tfstate -backend-config=resource_group_name=STRGRG -backend-config=subscription_id=STRGSUBID -backend-config=tenant_id=TENANTID -backend-config=client_id=SP_CLIENT_ID -backend-config=client_secret=SP_CLIENT_SECRET

Notice that you can change "STRGSUBID" to whatever you wish.