[Bug]: Timestream for InfluxDB resources cannot be imported without destruction

[ronanbarrett]

Terraform Core Version

1.8.1

AWS Provider Version

5.65.0

Affected Resource(s)

• aws_timestreaminfluxdb_db_instance

Expected Behavior

It should be possible to import an existing AWS Timestream Influx DB database without it being recreated and losing all the data in the database.

Actual Behavior

The terraform plan makes it clear after importing the resource the database will be destroyed and then recreated.

Relevant Error/Panic Output Snippet

Terraform will perform the following actions:

  # aws_timestreaminfluxdb_db_instance.redacted must be replaced
  # (imported from "redacted")
  # Warning: this will destroy the imported resource
-/+ resource "aws_timestreaminfluxdb_db_instance" "redacted" {
        allocated_storage                 = 20
      ~ arn                               = "redacted" -> (known after apply)
      ~ availability_zone                 = "eu-west-1b" -> (known after apply)
      + bucket                            = (sensitive value) # forces replacement
        db_instance_type                  = "db.influx.medium"
        db_parameter_group_identifier     = "redacted"
      ~ db_storage_type                   = "InfluxIOIncludedT1" -> (known after apply)
      ~ deployment_type                   = "SINGLE_AZ" -> (known after apply)
      ~ endpoint                          = "redacted" -> (known after apply)
      ~ id                                = "redacted-id" -> (known after apply)
      ~ influx_auth_parameters_secret_arn = "arn:aws:secretsmanager:eu-west-1:redacted:secret:redacted" -> (known after apply)
        name                              = "redacted"
      + organization                      = (sensitive value) # forces replacement
      + password                          = (sensitive value) # forces replacement
        publicly_accessible               = false
      + secondary_availability_zone       = (known after apply)
      + username                          = (sensitive value) # forces replacement
        vpc_security_group_ids            = [
            "redacted",
        ]
        vpc_subnet_ids                    = [
            "redacted",
            "redacted",
            "redacted",
        ]

        log_delivery_configuration {
            s3_configuration {
                bucket_name = "redacted"
                enabled     = true
            }
        }
    }

Plan: 1 to import, 1 to add, 0 to change, 1 to destroy.

Terraform Configuration Files

import {
  to = aws_timestreaminfluxdb_db_instance.redacted
  id = "redacted"
}

data "aws_secretsmanager_secret" "influx-secrets" {
  arn = "redacted"
}

data "aws_secretsmanager_secret_version" "influx-current" {
  secret_id = data.aws_secretsmanager_secret.influx-secrets.id
}

resource "aws_timestreaminfluxdb_db_instance" "redacted" {
  allocated_storage             = 20
  db_instance_type              = "db.influx.medium"
  vpc_subnet_ids                = redacted
  vpc_security_group_ids        = [redacted]
  name                          = "redacted"
  publicly_accessible           = false
  db_parameter_group_identifier = "redacted"
  username                      = jsondecode(data.aws_secretsmanager_secret_version.influx-current.secret_string)["username"]
  password                      = jsondecode(data.aws_secretsmanager_secret_version.influx-current.secret_string)["password"]
  organization                  = jsondecode(data.aws_secretsmanager_secret_version.influx-current.secret_string)["organization"]
  bucket                        = jsondecode(data.aws_secretsmanager_secret_version.influx-current.secret_string)["bucket"]

  log_delivery_configuration {
    s3_configuration {
      bucket_name = "redacted"
      enabled     = true
    }
  }
}

Steps to Reproduce

1. Create a TImestream InfluxDB in AWS using the CLI or Console
2. Try and import the state to terraform

Debug Output

No response

Panic Output

No response

Important Factoids

The fields organization, username, password, bucket are all marked as required so removing them is not an option.

The values I'm assigning to organization, username, password, bucket are the string values currently assigned to them.

References

PR that introduced support for the resource https://github.com/hashicorp/terraform-provider-aws/issues/36398

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Hey @ronanbarrett 👋 Thank you for taking the time to raise this! I'm reviewing this issue while triaging, and wanted to offer a potential workaround until this is able to be prioritized and more thoroughly addressed. You should be able to add bucket, organization, password, and username to ignore_changes and prevent this from occurring. It might also be worth reviewing your logs to see what's being returned for data.aws_secretsmanager_secret_version.influx-current and compare that to the logging for the aws_timestreaminfluxdb_db_instance resource.

If you're able to supply debug logging (redacted as needed), that may also help whoever ultimately picks this up to look into it.

[ronanbarrett]

Thanks @justinretzolk I have added the lifecycle rule as follows and it imports without destroying the resource 👍

  lifecycle {
    ignore_changes = [
      username,
      password,
      organization,
      bucket
    ]
  }

I can confirm that I had already tried setting the organization, username, password, bucket attributes to the literal string values they already has (not going via the secret) and terraform was still suggesting the destructive action. So it is still a bug.

[trevorbonas]

The problem is that aws_timestreaminfluxdb_db_instance solely uses the AWS SDK for Go and never uses the InfluxDB v2 API. username, password, organization, and bucket are required when creating the instance by the AWS SDK for Go but there are no AWS SDK for Go methods that allow the reading of these values. These values can only be read from the instance using the InfluxDB v2 API.

A workaround I added, intially, was to read from the secret that is created when the instance is created (internal/service/timestreaminfluxdb/db_instance.go line 529). This secret will contain the username, password, organization, and bucket values. However, this change was removed when the PR was merged, I'm not sure why.