Passing backend config through `TerraformVariable`s is not supported by Terraform

[tieto-hemmopaa]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

cdktf & Language Versions

Node v18.13.0 Terraform v1.3.4 cdktf 0.15.1

Affected Resource(s)

TerraformVariable

Debug Output

Expected Behavior

Creating a TerraformVariable should get the values of environment variables TF_VAR_awsAccessKey and TF_VAR_awsSecretAccessKey.

Running cdktf deploy test-stack-1 or cdktf deploy test-stack-2 should be successful.

main.ts:

import { Construct } from 'constructs';
import { App, TerraformStack, S3Backend, TerraformVariable } from 'cdktf';
import { AwsProvider } from './.gen/providers/aws/provider';
import { S3Bucket } from './.gen/providers/aws/s3-bucket';
import { DynamodbTable } from './.gen/providers/aws/dynamodb-table';

interface MultiStackConfig {
    environment: string;
}
class MultiStack extends TerraformStack {
  constructor(scope: Construct, id: string, config: MultiStackConfig) {
    super(scope, id);

    const awsAccessKey = new TerraformVariable(this, 'awsAccessKey', {
      type: 'string',
      description: 'AWS Access Key',
      sensitive: true
    })
    const awsSecretAccessKey = new TerraformVariable(this, 'awsSecretAccessKey', {
      type: 'string',
      description: 'AWS Secret Access Key',
      sensitive: true
    })
    new AwsProvider(this, 'aws', {
      region: 'eu-west-1',
      accessKey: awsAccessKey.value,
      secretKey: awsSecretAccessKey.value
    });

    const { environment } = config;

    new S3Bucket(this, 's3_backend_bucket', {
      bucket: `cdktf-remote-backend-${environment}`,
    });
    new DynamodbTable(this, 'dynamodb_lock_table', {
      name: `cdktf-remote-backend-lock-${environment}`,
      billingMode: 'PAY_PER_REQUEST',
      attribute: [
        {
          'name': 'LockID',
          'type': 'S'
        }
      ],
      hashKey: 'LockID',
    });
    new S3Backend(this, {
      bucket: `cdktf-remote-backend-${environment}`,
      key: `integration_portal/terraform_${environment}.tfstate`,
      encrypt: true,
      region: 'eu-west-1',
      dynamodbTable: `cdktf-remote-backend-lock-${environment}`,
      accessKey: awsAccessKey.value,
      secretKey: awsSecretAccessKey.value
    });
  }
}

const app = new App();
new MultiStack(app, 'test-stack-1', {
  environment: 'qa'
});
new MultiStack(app, 'test-stack-2', {
  environment: 'prod'
});
app.synth();

Environment variables:

TF_VAR_awsAccessKey='accessKeyValueForIAMWithPermissions'
TF_VAR_awsSecretAccessKey='secretAccessKeyValueForIAMWithPermissions'

Actual Behavior

cdktf deploy test-stack-1 prints the following output:

test-stack-1  Initializing the backend...

1 Stack deploying     0 Stacks done     0 Stacks waiting
[2023-01-25T14:29:22.234] [ERROR] default - ╷
│ Error: error configuring S3 Backend: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
│   status code: 403, request id: [some_id]
│ 
│ 
test-stack-1  ╷
                            │ Error: error configuring S3 Backend: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
                            │   status code: 403, request id: [some_id]
                            │ 
                            │ 
                            ╵

0 Stacks deploying     1 Stack done     0 Stacks waiting
Error: non-zero exit code 1

Steps to Reproduce

1. Create an AWS IAM User with the required permissions.

2. Make an empty directory and go there

   mkdir test-cdktf
   cd test-cdktf

3. Init the project:

   cdktf init --template=typescript

4. When done, modify the cdktf.json file by adding the provider:

   "terraformProviders": [
   "aws@~> 4.0"
   ]

5. Create the appropriate TypeScript classes for the provider (creates content to the .gen-folder):

   cdktf get

6. Copy the content of my main.ts file to your project.

7. Add TF_VAR_awsAccessKey and TF_VAR_awsSecretAccessKey to environment variables.

8. Deploy the test-stack-1 (or test-stack-2):

cdktf deploy test-stack-1

Important Factoids

The IAM role used for this deployment does have the correct permissions. The code works fine by hard-coding the environment variable values to the code and getting rid of TerraformVariable.

References

• 0000

[ansgarm]

Hi @tieto-hemmopaa 👋 Just to confirm that there's not a problem in setting the environment variable: Could you try to run echo $TF_VAR_awsAccessKey to confirm that the value of the variable does not contain quotes (') or similar?

[ansgarm]

I just checked the related Terraform docs, and it appears that you can't use Terraform Variables to configure backends.

We should catch such errors in the CDKTF core library (e.g. using an Aspect, as we already do for other checks) and print a helpful warning.

I'm wondering why Terraform isn't giving a better error, though 🤔

[tieto-hemmopaa]

Hi @ansgarm ! No quotes. I've tried setting the environment variables both with and without quotes. Neither of those add them to the value of the variable.

[ansgarm]

From the docs of the S3 backend:

This can also be sourced from the AWS_ACCESS_KEY_ID environment variable

Sounds like you should be able to pass the required keys using the environment variables defined in the docs of the S3 backend.

[tieto-hemmopaa]

Thank you so much @ansgarm !

In case anyone else is wondering, I was able to get my code working by first modifying the code in the following way:

new S3Backend(this, {
      bucket: `cdktf-remote-backend-${environment}`,
      key: `integration_portal/terraform_${environment}.tfstate`,
      encrypt: true,
      region: 'eu-west-1',
      dynamodbTable: `cdktf-remote-backend-lock-${environment}`,
      accessKey: process.env.TF_VAR_awsAccessKey,
      secretKey: process.env.TF_VAR_awsSecretAccessKey
    });

And running the terraform init -migrate-state command in cdktf.out/stacks/test-stack-1.

After that cdktf deploy test-stack-1 works perfectly.

[ansgarm]

Glad you figured it out! The only warning I have about using process.env.TF_VAR_awsAccessKey is that it will cause the secret to end up in the generated cdk.tf.json file. So be cautious if you commit that file to CI or do something else with it!

[tieto-hemmopaa]

I only use process.env.TF_VAR_awsAccessKey and process.env.TF_VAR_awsSecretAccessKey when configuring the S3Backend. 👍 For all the other classes I use the TerraformVariable.

The secrets used configuring the backend do not seem to end up in the cdk.tf.json files.

The content of the file for the Backend looks like this:

"backend": {
      "s3": {
        "bucket": "cdktf-remote-backend-qa",
        "dynamodb_table": "cdktf-remote-backend-lock-qa",
        "encrypt": true,
        "key": "integration_portal/terraform_qa.tfstate",
        "region": "eu-west-1"
      }
}

[ansgarm]

Hm, maybe you ran cdktf diff/plan/apply/deploy/destroy without those environment variables at a later stage? If you have them set, they should end up in the generated file. But if you run synth (happens automatically for the commands I mentioned previously) it'll recreate that generated file – thereby possibly removing those secrets again. It does not need to be a bad thing, I just wanted to point that out as it can bring trouble depending on when you set those secrets and what you do with the generated file afterwards.

[tieto-hemmopaa]

Sure, I ran all sorts of commands during the process. I'll keep an eye out later on 👍

[ansgarm]

This just came up in a question on the CDK Dev Slack where CDKTF allowed the value of a AWS SSM Parameter data source to be passed to a Terraform Backend although Terraform did not support this. This seemed to have resulted in an unresolved token that was put into the backend config.