Snowflake-Labs / terraform-provider-snowflake

Terraform provider for managing Snowflake accounts
https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest
MIT License
548 stars 419 forks source link

[Bug]: Table Column Updates are not recognized by Terraform Provider #3007

Closed JESCHO99 closed 2 days ago

JESCHO99 commented 2 months ago

Terraform CLI Version

1.5.7

Terraform Provider Version

0.94.1

Terraform Configuration

terraform {
  required_version = "~> 1.5.0"
  required_providers {
    snowflake = {
      source  = "Snowflake-Labs/snowflake"
      version = "~> 0.94.0"
    }
  }
}

provider "snowflake" {
  role = "SYSADMIN"
}

resource "snowflake_database" "this" {
  name = "TEST_DATABASE"
}

resource "snowflake_schema" "this" {
  database                    = snowflake_database.this.name
  name                        = "TEST_SCHEMA"
  is_transient                = "false"
  with_managed_access         = "false"
  data_retention_time_in_days = 1
}

resource "snowflake_table" "table_1" {
  database                    = snowflake_database.this.name
  schema                      = snowflake_schema.this.name
  name                        = "TEST_TABLE_V1"
  data_retention_time_in_days = 1
  change_tracking             = false

  column {
    name = "ID"
    type = "NUMBER"
  }

  column {
    name = "NAME"
    type = "VARCHAR(3)"
  }
}

resource "snowflake_table" "table_2" {
  database                    = snowflake_database.this.name
  schema                      = snowflake_schema.this.name
  name                        = "TEST_TABLE_V2"
  data_retention_time_in_days = 1
  change_tracking             = false

  column {
    name = "ID"
    type = "NUMBER(11,2)"
  }

  column {
    name = "NAME"
    type = "VARCHAR(255)"
  }
}

Category

category:resource

Object type(s)

resource:table

Expected Behavior

The code provided above could be applied to a Snowflake Account without any problem. If you change the data type of a column for example change the "type" of the column "NAME" in the snowflake table "TEST_TABLE_V2" to "VARCHAR(3)" the next terraform plan should recognize this change and provide a plan how this change could be applied to the backend.

resource "snowflake_table" "table_2" {
  database                    = snowflake_database.this.name
  schema                      = snowflake_schema.this.name
  name                        = "TEST_TABLE_V2"
  data_retention_time_in_days = 1
  change_tracking             = false

  column {
    name = "ID"
    type = "NUMBER(11,2)"
  }

  column {
    name = "NAME"
    type = "VARCHAR(3)"
  }
}

Actual Behavior

Even if the resource block for the snowflake table "table_2" is changed the terraform plan running on the changed code shows no changes. Terraform does not recognize that there should be a change on the data type of the column.

Steps to Reproduce

  1. Copy the following configuration

resource "snowflake_database" "this" { name = "TEST_DATABASE" }

resource "snowflake_schema" "this" { database = snowflake_database.this.name name = "TEST_SCHEMA" is_transient = "false" with_managed_access = "false" data_retention_time_in_days = 1 }

resource "snowflake_table" "table_1" { database = snowflake_database.this.name schema = snowflake_schema.this.name name = "TEST_TABLE_V1" data_retention_time_in_days = 1 change_tracking = false

column { name = "ID" type = "NUMBER" }

column { name = "NAME" type = "VARCHAR(3)" } }

resource "snowflake_table" "table_2" { database = snowflake_database.this.name schema = snowflake_schema.this.name name = "TEST_TABLE_V2" data_retention_time_in_days = 1 change_tracking = false

column { name = "ID" type = "NUMBER(11,2)" }

column { name = "NAME" type = "VARCHAR(255)" } }

  1. Run terraform apply to a Snowflake Account

  2. Change the resource block "snowflake_table" "table_2" like this:

resource "snowflake_table" "table_2" { database = snowflake_database.this.name schema = snowflake_schema.this.name name = "TEST_TABLE_V2" data_retention_time_in_days = 1 change_tracking = false

column { name = "ID" type = "NUMBER(11,2)" }

column { name = "NAME" type = "VARCHAR(3)" } }

  1. Run terraform plan
  2. You will see no changes in the terraform plan which is not correct

How much impact is this issue causing?

High

Logs

No response

Additional Information

It would be great if you could provide us with an estimation when this issue could be fixed. We tested around with various provider versions and found out that this issue appears the first time in version "0.90.0". In version "0.89.0" the terraform plan showed changes as expected.

We plan to have huge changes during the next three weeks. If it is not possible to provide a hotfix we need to downgrade our current terraform project. Thanks for your help!

Would you like to implement a fix?

sfc-gh-asawicki commented 2 months ago

Hey @JESCHO99. Thanks for raising this issue.

Unfortunately, this is a known error that affects multiple resources, not only tables (e.g., check https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/2735).

Long story short: data types with arguments (like VARCHAR(3) and VARCHAR(200)) are not always handled properly. In this case, they are both synonymized to check if the type changed, but their length is not taken into consideration, so the diff is suppressed incorrectly.

We should start working on procedures and functions next week, so the datatype topic will be addressed then. Before that, we can add a hotfix to handle properly at least changes to VARCHAR length in the table resource. The hotfix won't make it probably into the next release (tomorrow or on Friday) but should be ready next week.

sfc-gh-asawicki commented 2 months ago

I will also make sure it's added to the known issues/limitations in our FAQ (https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/CREATING_ISSUES.md#commonly-known-issues).

JESCHO99 commented 2 months ago

Hey @sfc-gh-asawicki, thanks for the reply. A Hotfix for Varchar Fields in the next week would help a lot. If possible it would be great if NUMBER fields could be added to the Hotfix too. Those are the two data types mainly used in our environments. Thanks for your help!

sfc-gh-asawicki commented 2 months ago

Hey @JESCHO99. I think we can include numbers, too; I will confirm next week.

sfc-gh-asawicki commented 2 months ago

Hey @JESCHO99.

I have prepared a fix both for text and number columns in https://github.com/Snowflake-Labs/terraform-provider-snowflake/pull/3020. It will be released as part of v0.95.0 today or tomorrow (we have to wrap a few other pull requests before making the release). Please check the migration guide for the description of the changes and report any problems with this hotfix.

sfc-gh-asawicki commented 2 months ago

Hey @JESCHO99, we had to postpone the v0.95.0 release to Monday, sorry for the inconvenience.

JESCHO99 commented 2 months ago

Hey @sfc-gh-asawicki , no problem and thanks for the information. As there was no release yesterday I just wanted to ask if it will come during this week or is planned much later right now?

sfc-gh-asawicki commented 2 months ago

Hey. Yeah, we have unfortunately faced a few unexpected issues with the underlying library while working on the fix (#3032) that we really want to include in this release. Currently, we estimate the release will happen late today or early tomorrow. I'll keep you posted.

sfc-gh-asawicki commented 1 month ago

Hey @JESCHO99 . We have just released v0.95.0 of the provider. It contains a hotfix for this issue. Please check and let us know if it works for you.

sfc-gh-asawicki commented 2 weeks ago

Hey @JESCHO99, did you validate the fix? If so, can we close the issue?

sfc-gh-asawicki commented 2 days ago

Closing as the fix was provided. Please file a new issue if there are any problems in the newest provider version.