fivetran / terraform-provider-fivetran

Terraform Provider for Fivetran
https://fivetran.com
Apache License 2.0
41 stars 22 forks source link

Connector throws inconsistent result in tf-cloud #278

Closed IkeHull closed 3 months ago

IkeHull commented 4 months ago

I get this error message from an "apply" run using TF-cloud

-- Error: Provider produced inconsistent result after apply When applying changes to fivetran_connector.connector_fivetran_log, provider "provider[\"registry.terraform.io/fivetran/fivetran\"]" produced an unexpected new value: .config: was absent, but now present.

This is a bug in the provider, which should be reported in the provider's own issue tracker.

TF code looks like this

resource "fivetran_destination" "destination" { group_id = fivetran_group.group.id region = var.fivetran_region run_setup_tests = false service = "big_query" time_zone_offset = var.fivetran_time_zone_offset

config { project_id = random_id.cm-project-id.hex data_set_location = var.fivetran_data_set_location } }

resource "fivetran_connector" "connector_fivetran_log" { group_id = fivetran_group.group.id service = "fivetran_log"

 destination_schema {
     name = "fivetran_log"
 }

}


TF version in cloud setup is 1.7.5 provider version is 1.1.17

beevital commented 3 months ago

@IkeHull as a WA just define config block for a connector. Looks like side-effect of import fix.

AaronCoquet-Easypark commented 3 months ago

I have a similar / the same issue with an SQL Server connection that I'm trying to import. Output:

Error: Provider produced inconsistent result after apply When applying changes to fivetranconnector.ext[redacted], provider "provider[\"registry.terraform.io/fivetran/fivetran\"]" produced an unexpected new value: .config.username: was cty.StringVal([redacted]), but now null. This is a bug in the provider, which should be reported in the provider's own issue tracker.

Code that produced this:

resource "fivetran_connector" "ext_<redacted>" {
  group_id        = <redacted>
  run_setup_tests = false
  service         = "sql_server"

  config {
    update_method = "TELEPORT"
    database = <redacted>
    password = <redacted>
    port     = <redacted>
    host     = <redacted>
    username = <redacted>
  }

  destination_schema {
    prefix = <redacted>
  }
}

I'm not sure if it matters, but the terraform plan shows this: image

TF version: Terraform v1.7.5 on darwin_arm64

beevital commented 3 months ago

Fixed in v1.1.20

AaronCoquet-Easypark commented 3 months ago

I'm afraid it hasn't been fixed. I should mention that I'm not using Terraform Cloud, just Terraform.

My output (redacted)

``` > terraform init -upgrade Initializing the backend... Initializing provider plugins... - Finding fivetran/fivetran versions matching ">= 1.0.0"... - Installing fivetran/fivetran v1.1.20... - Installed fivetran/fivetran v1.1.20 (signed by a HashiCorp partner, key ID BD40EF4531102765) Partner and community providers are signed by their developers. If you'd like to know more about provider signing, you can read about it here: https://www.terraform.io/docs/cli/plugins/signing.html Terraform has made some changes to the provider dependency selections recorded in the .terraform.lock.hcl file. Review those changes and commit them to your version control system if they represent changes you intended to make. Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. > terraform apply Acquiring state lock. This may take a few moments... fivetran_connector.ext_visma: Refreshing state... [id=] Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: ~ update in-place Terraform will perform the following actions: # fivetran_connector.ext_ will be updated in-place ~ resource "fivetran_connector" "ext_" { ~ connected_by = "" -> (known after apply) ~ created_at = "2024-02-03 06:26:17.9696 +0000 UTC" -> (known after apply) ~ id = "" -> (known after apply) ~ name = "" -> (known after apply) + trust_certificates = (known after apply) + trust_fingerprints = (known after apply) # (3 unchanged attributes hidden) ~ config { + abs_connection_string = (known after apply) + abs_container_name = (known after apply) + access_key = (known after apply) + access_key_id = (known after apply) + access_token = (sensitive value) + account = (known after apply) + account_id = (known after apply) + account_key = (sensitive value) + account_region = (known after apply) + account_type = (known after apply) + action_report_time = (known after apply) + agent_config_method = (known after apply) + agent_host = (known after apply) + agent_ora_home = (known after apply) + agent_password = (sensitive value) + agent_port = (known after apply) + agent_public_cert = (known after apply) + agent_user = (known after apply) + aggregation = (known after apply) + always_encrypted = (known after apply) + api_access_token = (sensitive value) + api_environment = (known after apply) + api_key = (sensitive value) + api_quota = (known after apply) + api_requests_per_minute = (known after apply) + api_secret = (sensitive value) + api_token = (sensitive value) + api_type = (known after apply) + api_url = (known after apply) + api_version = (known after apply) + app_sync_mode = (known after apply) + append_file_option = (known after apply) + archive_pattern = (known after apply) + are_soap_credentials_provided = (known after apply) + asm_option = (known after apply) + asm_oracle_home = (known after apply) + asm_password = (sensitive value) + asm_tns = (known after apply) + asm_user = (known after apply) + auth = (known after apply) + auth_environment = (known after apply) + auth_mode = (known after apply) + auth_type = (known after apply) + authentication_method = (known after apply) + authorization_method = (known after apply) + aws_region_code = (known after apply) + base_url = (known after apply) + blockchain = (known after apply) + bucket = (known after apply) + bucket_name = (known after apply) + bucket_service = (known after apply) + certificate = (known after apply) + click_attribution_window = (known after apply) + client_id = (known after apply) + client_name = (known after apply) + client_secret = (sensitive value) + cloud_storage_type = (known after apply) + company_id = (known after apply) + compression = (known after apply) + config_method = (known after apply) + config_type = (known after apply) + connection_method = (known after apply) + connection_string = (known after apply) + connection_type = (known after apply) + consumer_group = (known after apply) + consumer_key = (sensitive value) + consumer_secret = (sensitive value) + container_name = (known after apply) + conversion_report_time = (known after apply) + conversion_window_size = (known after apply) + csv_definition = (known after apply) + customer_id = (known after apply) + customer_list_id = (known after apply) + daily_api_call_limit = (known after apply) + data_access_method = (known after apply) + data_center = (known after apply) + dataset_id = (known after apply) + datasource = (known after apply) + date_granularity = (known after apply) + delimiter = (known after apply) + direct_capture_method = (known after apply) + distributed_connector_cluster_size = (known after apply) + domain = (known after apply) + domain_host_name = (known after apply) + domain_name = (known after apply) + domain_type = (known after apply) + email = (known after apply) + empty_header = (known after apply) + enable_all_dimension_combinations = (known after apply) + enable_archive_log_only = (known after apply) + enable_data_extensions_syncing = (known after apply) + enable_distributed_connector_mode = (known after apply) + enable_enrichments = (known after apply) + enable_exports = (known after apply) + enable_tde = (known after apply) + encryption_key = (sensitive value) + endpoint = (known after apply) + engagement_attribution_window = (known after apply) + entity_id = (known after apply) + environment = (known after apply) + escape_char = (known after apply) + escape_char_options = (known after apply) + eu_region = (known after apply) + export_storage_type = (known after apply) + external_id = (known after apply) + file_type = (known after apply) + finance_account_sync_mode = (known after apply) + folder_id = (known after apply) + ftp_host = (known after apply) + ftp_password = (sensitive value) + ftp_port = (known after apply) + ftp_user = (known after apply) + function = (known after apply) + function_app = (known after apply) + function_key = (known after apply) + function_name = (known after apply) + function_trigger = (sensitive value) + gcs_bucket = (known after apply) + gcs_folder = (known after apply) + group_name = (known after apply) + hana_mode = (known after apply) + has_manage_permissions = (known after apply) + home_folder = (known after apply) + identity = (known after apply) + include_ocapi_endpoints = (known after apply) + instance = (known after apply) + integration_key = (known after apply) + is_account_level_connector = (known after apply) + is_auth2_enabled = (known after apply) + is_custom_api_credentials = (known after apply) + is_external_activities_endpoint_selected = (known after apply) + is_ftps = (known after apply) + is_keypair = (known after apply) + is_multi_entity_feature_enabled = (known after apply) + is_new_package = (known after apply) + is_private_key_encrypted = (known after apply) + is_private_link_required = (known after apply) + is_public = (known after apply) + is_sailthru_connect_enabled = (known after apply) + is_secure = (known after apply) + is_sftp_creds_available = (known after apply) + is_single_table_mode = (known after apply) + is_vendor = (known after apply) + key = (known after apply) + last_synced_changes__utc_ = (known after apply) + latest_version = (known after apply) + limit_for_api_calls_to_external_activities_endpoint = (known after apply) + list_strategy = (known after apply) + login_password = (sensitive value) + merchant_id = (known after apply) + message_type = (known after apply) + named_range = (known after apply) + network_code = (known after apply) + non_standard_escape_char = (known after apply) + null_sequence = (known after apply) + oauth_token = (sensitive value) + oauth_token_secret = (sensitive value) + on_error = (known after apply) + on_premise = (known after apply) + organization = (known after apply) + organization_id = (known after apply) + passphrase = (sensitive value) + pat = (sensitive value) + path = (known after apply) + pattern = (known after apply) + pdb_name = (known after apply) + pem_certificate = (sensitive value) + post_click_attribution_window_size = (known after apply) + prebuilt_report = (known after apply) + prefix = (known after apply) + private_key = (sensitive value) + product = (known after apply) + project_id = (known after apply) + public_key = (known after apply) + publication_name = (known after apply) + pull_archived_campaigns = (known after apply) + query_id = (known after apply) + region = (known after apply) + region_api_url = (known after apply) + region_auth_url = (known after apply) + region_token_url = (known after apply) + replica_id = (known after apply) + replication_slot = (known after apply) + report_type = (known after apply) + report_url = (known after apply) + resource_url = (known after apply) + rest_api_limit = (known after apply) + role = (known after apply) + role_arn = (sensitive value) + rollback_window_size = (known after apply) + s3bucket = (known after apply) + s3external_id = (known after apply) + s3folder = (known after apply) + s3role_arn = (sensitive value) + sales_account_sync_mode = (known after apply) + sap_user = (known after apply) + secret = (sensitive value) + secret_key = (sensitive value) + secrets = (sensitive value) + security_protocol = (known after apply) + server = (known after apply) + server_url = (known after apply) + service_version = (known after apply) + sftp_host = (known after apply) + sftp_is_key_pair = (known after apply) + sftp_password = (sensitive value) + sftp_port = (known after apply) + sftp_user = (known after apply) + share_url = (known after apply) + sheet_id = (known after apply) + shop = (known after apply) + short_code = (sensitive value) + should_sync_events_with_deleted_profiles = (known after apply) + show_records_with_no_metrics = (known after apply) + sid = (known after apply) + site_id = (known after apply) + skip_after = (known after apply) + skip_before = (known after apply) + soap_uri = (known after apply) + source = (known after apply) + sub_domain = (known after apply) + subdomain = (known after apply) + subscriber_name = (known after apply) + support_connected_accounts_sync = (known after apply) + support_nested_columns = (known after apply) + swipe_attribution_window = (known after apply) + sync_data_locker = (known after apply) + sync_format = (known after apply) + sync_formula_fields = (known after apply) + sync_metadata = (known after apply) + sync_method = (known after apply) + sync_mode = (known after apply) + sync_mode_advertiser = (known after apply) + sync_mode_seat = (known after apply) + sync_multiple_accounts = (known after apply) + sync_pack_mode = (known after apply) + sync_pull_api = (known after apply) + sync_type = (known after apply) + target_entity_id = (known after apply) + technical_account_id = (known after apply) + test_table_name = (known after apply) + time_zone = (known after apply) + timeframe_months = (known after apply) + tns = (known after apply) + token_key = (sensitive value) + token_secret = (sensitive value) + tunnel_host = (known after apply) + tunnel_port = (known after apply) + tunnel_user = (known after apply) + unique_id = (known after apply) + update_config_on_each_sync = (known after apply) + uri = (known after apply) + url_format = (known after apply) + use_api_keys = (known after apply) + use_customer_bucket = (known after apply) + use_oracle_rac = (known after apply) + use_pgp_encryption_options = (known after apply) + use_service_account = (known after apply) + use_template_labels = (known after apply) + use_webhooks = (known after apply) + use_workspace = (known after apply) + user = (known after apply) + user_id = (known after apply) + user_key = (known after apply) + user_name = (known after apply) + username = "" + view_attribution_window = (known after apply) + view_through_attribution_window_size = (known after apply) + workspace_same_as_source = (known after apply) # (5 unchanged attributes hidden) } # (1 unchanged block hidden) } Plan: 0 to add, 1 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes fivetran_connector.ext_: Modifying... [id=] ╷ │ Error: Provider produced inconsistent result after apply │ │ When applying changes to fivetran_connector.ext_, provider "provider[\"registry.terraform.io/fivetran/fivetran\"]" produced an unexpected new │ value: .config.username: was cty.StringVal(""), but now null. │ │ This is a bug in the provider, which should be reported in the provider's own issue tracker. ╵ Releasing state lock. This may take a few moments... ```

beevital commented 3 months ago

@AaronCoquet-Easypark that's another one. I'll take a look. From what version you're upgrading?

beevital commented 3 months ago

Looks like a problem with sql_server API response. It doesn't return username for some reason and it came empty into state. Let me dig into it.

beevital commented 3 months ago

And the most important - why are you're using username field? You should use just user. https://registry.terraform.io/providers/fivetran/fivetran/latest/docs/resources/connector#user What is happening:

beevital commented 3 months ago

@AaronCoquet-Easypark you need to update your config the following way:


resource "fivetran_connector" "ext_<redacted>" {
  group_id        = <redacted>
  run_setup_tests = false
  service         = "sql_server"

  config {
    update_method = "TELEPORT"
    database = <redacted>
    password = <redacted>
    port     = <redacted>
    host     = <redacted>
    user = <redacted>
  }

  destination_schema {
    prefix = <redacted>
  }
}
beevital commented 3 months ago

It seems like it's hard to find a set of fields expected for particular connector.

We could strictly check if only expected fields are configured on plan so provider will return an error on attempt to configure username instead of user for sql_server.

Also we will prepare an example of configuration for every connector to make it easier to ensure you're configuring it correctly. Stay tuned.

AaronCoquet-Easypark commented 3 months ago

Thanks for all the feedback, and the expected updates!