fivetran / terraform-provider-fivetran

Terraform Provider for Fivetran
https://fivetran.com
Apache License 2.0
40 stars 23 forks source link

State issue for fivetran_connector with 0.7.2 #168

Closed jramos closed 1 year ago

jramos commented 1 year ago

After upgrading from 0.7.1 to 0.7.2, attempting to show the terraform state for our fivetran_connector results in an error.

$ terraform state show fivetran_connector.postgres
missing expected [
# fivetran_connector.postgres:
resource "fivetran_connector" "postgres" {

This works as expected with 0.7.1.

beevital commented 1 year ago

@jramos did you follow migration guide https://registry.terraform.io/providers/fivetran/fivetran/latest/docs/guides/version_update_guides#migration-guide? Or just updated the version in .tf file?

ann8ty commented 1 year ago

I have this problem too. I had been on 0.7.2-pre and went to 0.7.2 today.

I had already updated / split the schedule out https://registry.terraform.io/providers/fivetran/fivetran/latest/docs/guides/version_update_guides#migration-guide

and this morning I reran terraform init -upgrade

previously working on 0.7.2-pre is not working on 0.7.2. i went back to 0.7.2-pre and that is still working.

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: missing expected [
│ 
│   with fivetran_connector.braze_connector,
│   on braze.tf line 44, in resource "fivetran_connector" "braze_connector":
│   44:   config {
│ 
╵
╷
│ Error: missing expected [
│ 
│   with fivetran_connector.dynamodb_connector,
│   on dynamo.tf line 14, in resource "fivetran_connector" "dynamodb_connector":
│   14:   config {
│ 

the ddb connector

resource "fivetran_connector" "dynamodb_connector" {
  group_id           = data.aws_ssm_parameter.fivetran_group_id_external_id.value
  service            = "dynamodb"
  run_setup_tests    = true
  trust_fingerprints = true
  trust_certificates = true

  destination_schema {
    name = local.dynamodb_connector_name
  }

  config {
    aws_region_code = var.app_data_mapping.app_aws_region
    role_arn = "arn:aws:iam::${var.app_data_mapping.app_aws_account}:role/data/${var.role-name}"
  }
}

the braze config

resource "fivetran_connector" "braze_connector" {
  group_id           = data.aws_ssm_parameter.fivetran_group_id_external_id.value
  service            = "braze"
  run_setup_tests    = true
  trust_fingerprints = true
  trust_certificates = true

  destination_schema {
    name = local.braze_connector_name
  }

  config {
    s3bucket            = data.aws_ssm_parameter.fivetran_braze_bucket_name.value
    cloud_storage_type  = "AWS_S3"
    export_storage_type = "AWS_S3"
    s3folder            = "currents"
    # enable_exports = true # option dne
    api_url = "https://rest.iad-05.braze.com"
    api_key = data.aws_secretsmanager_secret_version.fivetran_to_braze_secret_value.secret_string
    s3external_id = data.aws_ssm_parameter.fivetran_group_id_external_id.value
    s3role_arn    = data.aws_ssm_parameter.fivetran_braze_environment_role_arn.value
  }

}
beevital commented 1 year ago

@ann8ty

and this morning I reran terraform init -upgrade

so you were already on v0.7.2-pre, but re-ran terraform init -upgrade once more? There was no need to do that actually, there's no schema breaking changes between this versions.

Anyway - it looks like v0.7.2 has some bugs, will investigate

beevital commented 1 year ago

@ann8ty UPD: reproduced the issue, investigating! Thank you for detailed description and example

beevital commented 1 year ago

@jramos @ann8ty UPD: Found the issue - the problem is with "fivetran_connector.config.servers" field. It's type was changed from string to set of strings, but state migration not prepared in code.

To fix the issue you can manually update .tfstate file, replace the following lines:

"resources": [
    {
      "mode": "managed",
      "type": "fivetran_connector",
      "name": "connector_name",
      "provider": "provider[\"registry.terraform.io/fivetran/fivetran\"]",
      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "auth": [],
            "config": [
              {
               ....
               "servers": "" -> "servers": null
              ....
              }
              ...

for ALL connector resources defined in your config.

Another way is to de-grade back to v0.7.2-pre and wait for new v0.7.3 with fixed migration and migrate on-to v0.7.3 directly.

This won't affect those who created a new configuration using v0.7.2 - they will be able to migrate to v0.7.3 also without any issues.

beevital commented 1 year ago

Released v0.7.3

ann8ty commented 1 year ago

v0.7.3 seems to be working nicely, thanks @beevital !

jramos commented 1 year ago

Same here, thanks @beevital!!!