fivetran / terraform-provider-fivetran

Terraform Provider for Fivetran
https://fivetran.com
Apache License 2.0
40 stars 23 forks source link

Error: Provider produced inconsistent result after apply (.config.folder_path:) #266

Closed NickMUnlockHealth closed 6 months ago

NickMUnlockHealth commented 7 months ago

Describe the bug

module.foo.fivetran_connector.this: Creating...
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to module.foo.fivetran_connector.this, provider
│ "provider[\"registry.terraform.io/fivetran/fivetran\"]" produced an unexpected new value: .config.folder_path: was
│ cty.StringVal("mypath"), but now null.

To Reproduce Provide an example .tf configuration

resource "fivetran_connector" "this" {
  group_id = data.fivetran_destination.this.id
  service  = "s3"

  auth {
    role_arn = aws_iam_role.this.arn
  }

  destination_schema {
    name = "s3"
    table = var.fivetran_destination_table
  }

  config {
    bucket      = var.s3_bucket
    folder_path = var.s3_prefix
    on_error = var.fivetran_on_error
  }
}

Expected behavior Connector resource creates without error.

Logs & Output

  # module.foo.fivetran_connector.this is tainted, so must be replaced
-/+ resource "fivetran_connector" "this" {
      ~ connected_by       = "REDACTED" -> (known after apply)
      ~ created_at         = "2024-03-05 17:54:20.914676 +0000 UTC" -> (known after apply)
      ~ id                 = "REDACTED" -> (known after apply)
      ~ name               = "s3.foo" -> (known after apply)
      ~ run_setup_tests    = false -> (known after apply)
      ~ trust_certificates = false -> (known after apply)
      ~ trust_fingerprints = false -> (known after apply)
        # (2 unchanged attributes hidden)

      ~ config {
          + abs_connection_string                               = (known after apply)
          + abs_container_name                                  = (known after apply)
          + access_key                                          = (known after apply)
          + access_key_id                                       = (known after apply)
          + access_token                                        = (sensitive value)
          + account                                             = (known after apply)
          + account_id                                          = (known after apply)
          + account_key                                         = (sensitive value)
          + account_type                                        = (known after apply)
          + action_report_time                                  = (known after apply)
          + agent_config_method                                 = (known after apply)
          + agent_host                                          = (known after apply)
          + agent_ora_home                                      = (known after apply)
          + agent_password                                      = (sensitive value)
          + agent_port                                          = (known after apply)
          + agent_public_cert                                   = (known after apply)
          + agent_user                                          = (known after apply)
          + aggregation                                         = (known after apply)
          + always_encrypted                                    = (known after apply)
          + api_access_token                                    = (sensitive value)
          + api_environment                                     = (known after apply)
          + api_key                                             = (sensitive value)
          + api_quota                                           = (known after apply)
          + api_requests_per_minute                             = (known after apply)
          + api_secret                                          = (sensitive value)
          + api_token                                           = (sensitive value)
          + api_type                                            = (known after apply)
          + api_url                                             = (known after apply)
          + api_version                                         = (known after apply)
          + app_sync_mode                                       = (known after apply)
          + append_file_option                                  = (known after apply)
          + archive_pattern                                     = (known after apply)
          + are_soap_credentials_provided                       = (known after apply)
          + asm_option                                          = (known after apply)
          + asm_oracle_home                                     = (known after apply)
          + asm_password                                        = (sensitive value)
          + asm_tns                                             = (known after apply)
          + asm_user                                            = (known after apply)
          + auth                                                = (known after apply)
          + auth_environment                                    = (known after apply)
          + auth_mode                                           = (known after apply)
          + auth_type                                           = (known after apply)
          + authorization_method                                = (known after apply)
          + aws_region_code                                     = (known after apply)
          + base_url                                            = (known after apply)
          + blockchain                                          = (known after apply)
          + bucket_name                                         = (known after apply)
          + bucket_service                                      = (known after apply)
          + certificate                                         = (known after apply)
          + click_attribution_window                            = (known after apply)
          + client_id                                           = (known after apply)
          + client_name                                         = (known after apply)
          + client_secret                                       = (sensitive value)
          + cloud_storage_type                                  = (known after apply)
          + company_id                                          = (known after apply)
          + compression                                         = (known after apply)
          + config_method                                       = (known after apply)
          + config_type                                         = (known after apply)
          + connection_method                                   = (known after apply)
          + connection_string                                   = (known after apply)
          + connection_type                                     = (known after apply)
          + consumer_group                                      = (known after apply)
          + consumer_key                                        = (sensitive value)
          + consumer_secret                                     = (sensitive value)
          + container_name                                      = (known after apply)
          + conversion_report_time                              = (known after apply)
          + conversion_window_size                              = (known after apply)
          + csv_definition                                      = (known after apply)
          + customer_id                                         = (known after apply)
          + customer_list_id                                    = (known after apply)
          + daily_api_call_limit                                = (known after apply)
          + data_access_method                                  = (known after apply)
          + data_center                                         = (known after apply)
          + database                                            = (known after apply)
          + dataset_id                                          = (known after apply)
          + datasource                                          = (known after apply)
          + date_granularity                                    = (known after apply)
          + delimiter                                           = (known after apply)
          + distributed_connector_cluster_size                  = (known after apply)
          + domain                                              = (known after apply)
          + domain_host_name                                    = (known after apply)
          + domain_name                                         = (known after apply)
          + domain_type                                         = (known after apply)
          + email                                               = (known after apply)
          + empty_header                                        = (known after apply)
          + enable_all_dimension_combinations                   = (known after apply)
          + enable_archive_log_only                             = (known after apply)
          + enable_data_extensions_syncing                      = (known after apply)
          + enable_distributed_connector_mode                   = (known after apply)
          + enable_enrichments                                  = (known after apply)
          + enable_exports                                      = (known after apply)
          + enable_tde                                          = (known after apply)
          + encryption_key                                      = (sensitive value)
          + endpoint                                            = (known after apply)
          + engagement_attribution_window                       = (known after apply)
          + entity_id                                           = (known after apply)
          + environment                                         = (known after apply)
          + escape_char                                         = (known after apply)
          + escape_char_options                                 = (known after apply)
          + eu_region                                           = (known after apply)
          + export_storage_type                                 = (known after apply)
          + external_id                                         = (known after apply)
          + file_type                                           = (known after apply)
          + finance_account_sync_mode                           = (known after apply)
          + folder_id                                           = (known after apply)
          + folder_path                                         = "mypath"
          + ftp_host                                            = (known after apply)
          + ftp_password                                        = (sensitive value)
          + ftp_port                                            = (known after apply)
          + ftp_user                                            = (known after apply)
          + function                                            = (known after apply)
          + function_app                                        = (known after apply)
          + function_key                                        = (known after apply)
          + function_name                                       = (known after apply)
          + function_trigger                                    = (sensitive value)
          + gcs_bucket                                          = (known after apply)
          + gcs_folder                                          = (known after apply)
          + group_name                                          = (known after apply)
          + hana_mode                                           = (known after apply)
          + has_manage_permissions                              = (known after apply)
          + home_folder                                         = (known after apply)
          + host                                                = (known after apply)
          + identity                                            = (known after apply)
          + include_ocapi_endpoints                             = (known after apply)
          + instance                                            = (known after apply)
          + integration_key                                     = (known after apply)
          + is_account_level_connector                          = (known after apply)
          + is_auth2_enabled                                    = (known after apply)
          + is_custom_api_credentials                           = (known after apply)
          + is_external_activities_endpoint_selected            = (known after apply)
          + is_ftps                                             = (known after apply)
          + is_keypair                                          = (known after apply)
          + is_multi_entity_feature_enabled                     = (known after apply)
          + is_new_package                                      = (known after apply)
          + is_private_key_encrypted                            = (known after apply)
          + is_private_link_required                            = (known after apply)
          + is_public                                           = (known after apply)
          + is_sailthru_connect_enabled                         = (known after apply)
          + is_secure                                           = (known after apply)
          + is_single_table_mode                                = (known after apply)
          + is_vendor                                           = (known after apply)
          + key                                                 = (known after apply)
          + last_synced_changes__utc_                           = (known after apply)
          + latest_version                                      = (known after apply)
          + limit_for_api_calls_to_external_activities_endpoint = (known after apply)
          + list_strategy                                       = (known after apply)
          + login_password                                      = (sensitive value)
          + merchant_id                                         = (known after apply)
          + message_type                                        = (known after apply)
          + named_range                                         = (known after apply)
          + network_code                                        = (known after apply)
          + non_standard_escape_char                            = (known after apply)
          + null_sequence                                       = (known after apply)
          + oauth_token                                         = (sensitive value)
          + oauth_token_secret                                  = (sensitive value)
          + on_premise                                          = (known after apply)
          + organization                                        = (known after apply)
          + organization_id                                     = (known after apply)
          + passphrase                                          = (sensitive value)
          + password                                            = (sensitive value)
          + pat                                                 = (sensitive value)
          + path                                                = (known after apply)
          + pattern                                             = (known after apply)
          + pdb_name                                            = (known after apply)
          + pem_certificate                                     = (sensitive value)
          + port                                                = (known after apply)
          + post_click_attribution_window_size                  = (known after apply)
          + prebuilt_report                                     = (known after apply)
          + prefix                                              = (known after apply)
          + private_key                                         = (sensitive value)
          + product                                             = (known after apply)
          + project_id                                          = (known after apply)
          + public_key                                          = (known after apply)
          + publication_name                                    = (known after apply)
          + pull_archived_campaigns                             = (known after apply)
          + query_id                                            = (known after apply)
          + region                                              = (known after apply)
          + region_api_url                                      = (known after apply)
          + region_auth_url                                     = (known after apply)
          + region_token_url                                    = (known after apply)
          + replica_id                                          = (known after apply)
          + replication_slot                                    = (known after apply)
          + report_type                                         = (known after apply)
          + report_url                                          = (known after apply)
          + resource_url                                        = (known after apply)
          + rest_api_limit                                      = (known after apply)
          + role                                                = (known after apply)
          + role_arn                                            = (sensitive value)
          + rollback_window_size                                = (known after apply)
          + s3bucket                                            = (known after apply)
          + s3external_id                                       = (known after apply)
          + s3folder                                            = (known after apply)
          + s3role_arn                                          = (sensitive value)
          + sales_account_sync_mode                             = (known after apply)
          + sap_user                                            = (known after apply)
          + secret                                              = (sensitive value)
          + secret_key                                          = (sensitive value)
          + secrets                                             = (sensitive value)
          + security_protocol                                   = (known after apply)
          + server                                              = (known after apply)
          + server_url                                          = (known after apply)
          + service_version                                     = (known after apply)
          + sftp_host                                           = (known after apply)
          + sftp_is_key_pair                                    = (known after apply)
          + sftp_password                                       = (sensitive value)
          + sftp_port                                           = (known after apply)
          + sftp_user                                           = (known after apply)
          + share_url                                           = (known after apply)
          + sheet_id                                            = (known after apply)
          + shop                                                = (known after apply)
          + short_code                                          = (sensitive value)
          + should_sync_events_with_deleted_profiles            = (known after apply)
          + show_records_with_no_metrics                        = (known after apply)
          + sid                                                 = (known after apply)
          + site_id                                             = (known after apply)
          + skip_after                                          = (known after apply)
          + skip_before                                         = (known after apply)
          + soap_uri                                            = (known after apply)
          + source                                              = (known after apply)
          + sub_domain                                          = (known after apply)
          + subdomain                                           = (known after apply)
          + subscriber_name                                     = (known after apply)
          + support_connected_accounts_sync                     = (known after apply)
          + support_nested_columns                              = (known after apply)
          + swipe_attribution_window                            = (known after apply)
          + sync_data_locker                                    = (known after apply)
          + sync_format                                         = (known after apply)
          + sync_formula_fields                                 = (known after apply)
          + sync_metadata                                       = (known after apply)
          + sync_method                                         = (known after apply)
          + sync_mode                                           = (known after apply)
          + sync_mode_advertiser                                = (known after apply)
          + sync_mode_seat                                      = (known after apply)
          + sync_multiple_accounts                              = (known after apply)
          + sync_pack_mode                                      = (known after apply)
          + sync_pull_api                                       = (known after apply)
          + sync_type                                           = (known after apply)
          + target_entity_id                                    = (known after apply)
          + technical_account_id                                = (known after apply)
          + test_table_name                                     = (known after apply)
          + time_zone                                           = (known after apply)
          + timeframe_months                                    = (known after apply)
          + tns                                                 = (known after apply)
          + token_key                                           = (sensitive value)
          + token_secret                                        = (sensitive value)
          + tunnel_host                                         = (known after apply)
          + tunnel_port                                         = (known after apply)
          + tunnel_user                                         = (known after apply)
          + unique_id                                           = (known after apply)
          + update_config_on_each_sync                          = (known after apply)
          + update_method                                       = (known after apply)
          + uri                                                 = (known after apply)
          + url_format                                          = (known after apply)
          + use_api_keys                                        = (known after apply)
          + use_customer_bucket                                 = (known after apply)
          + use_oracle_rac                                      = (known after apply)
          + use_pgp_encryption_options                          = (known after apply)
          + use_service_account                                 = (known after apply)
          + use_template_labels                                 = (known after apply)
          + use_webhooks                                        = (known after apply)
          + use_workspace                                       = (known after apply)
          + user                                                = (known after apply)
          + user_id                                             = (known after apply)
          + user_key                                            = (known after apply)
          + user_name                                           = (known after apply)
          + username                                            = (known after apply)
          + view_attribution_window                             = (known after apply)
          + view_through_attribution_window_size                = (known after apply)
          + workspace_same_as_source                            = (known after apply)
            # (2 unchanged attributes hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plugin version:

  fivetran = {
      source  = "fivetran/fivetran"
      version = "1.1.13"
    }

Additional context Connector works after manually adding in the folder path in the FiveTran UI, however the Terraform resource remains tainted.

beevital commented 6 months ago

Hey @NickMUnlockHealth! I've checked our public documentation about S3 connector and found out that you need to use prefix field to provide folder_path for s3 connectors: https://fivetran.com/docs/rest-api/connectors/config#amazons3

image image

I'm sorry about that inconsistency in our file connectors, but looks like it's historical thing...

Just update your config the following way:

resource "fivetran_connector" "this" {
 group_id = data.fivetran_destination.this.id
 service  = "s3"

 auth {
   role_arn = aws_iam_role.this.arn
 }

 destination_schema {
   name = "s3"
   table = var.fivetran_destination_table
 }

 config {
   bucket      = var.s3_bucket
   prefix = var.s3_prefix
   on_error = var.fivetran_on_error
 }
}

I've tested proposed solution and succesfully set Folder Path to desired value:

image