Snowflake-Labs / terraform-provider-snowflake

Terraform provider for managing Snowflake accounts
https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest
MIT License
532 stars 413 forks source link

snowflake_stage recreates every apply regardless of an actual change #2679

Open bmhah opened 5 months ago

bmhah commented 5 months ago

Terraform CLI and Provider Versions

v1.5.0 0.85.0

Terraform Configuration

resource "snowflake_stage" "LANDING_8X8_STAGE_INBOUND_CALLS_S3" {
  name                = "LANDING_8X8_STAGE_INBOUND_CALLS"
  url                 = var.s3_uri_8x8_inbound_calls
  database            = "AWS_LANDING_INGEST_DB_${var.SF_ENVIRONMENT}"
  schema              = "LANDING_8X8_SCHEMA"
  comment             = "Landing stage for inbound calls s3 bucket"
  file_format         = "FORMAT_NAME = AWS_LANDING_INGEST_DB_${var.SF_ENVIRONMENT}.LANDING_8X8_SCHEMA.LANDING_FF_CSV"
  copy_options        = "ON_ERROR = 'CONTINUE'"
  storage_integration = var.aws_storage_integration
}

Expected Behavior

Stage doesn't destroy and recreate

Actual Behavior

Stage recreates, this causes snowflake pipes to break having a status STOPPED_STAGE_DROPPED

Steps to Reproduce

  1. terraform apply

How much impact is this issue causing?

High

Logs

No response

Additional Information

No response

sfc-gh-asawicki commented 5 months ago

Hey @bmhah. Thanks for reaching out to us.

Please share the terraform run logs with TF_LOG=DEBUG enabled.

sfc-gh-bhill commented 5 months ago

Here is the file that the customer provided me. plan.txt.gz

bmhah commented 5 months ago

Any updates here?

sfc-gh-asawicki commented 5 months ago

Hey @bmhah. I will confirm in the morning, but it looks like an easy fix, so we may even be able to include it in tomorrow's release. I will keep you posted.

sfc-gh-asawicki commented 5 months ago

Hey @bmhah. We have released the new version: https://github.com/Snowflake-Labs/terraform-provider-snowflake/releases/tag/v0.89.0. Please bump and check if the error persists. Please remember to use the migration guide during version upgrades: https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/MIGRATION_GUIDE.md#migration-guide.

bmhah commented 5 months ago

Sounds good, we'll get to testing this later this week. I'll keep this issue open in the meantime

sfc-gh-asawicki commented 4 months ago

Hey @bmhah. Have you have a chance to test the newer versions after the introduced fixes?

bmhah commented 3 months ago

Yes, we've done the provider upgrade to 88 and have also used others, they haven't worked for us, the stage for this pipe still drops and recreates causing the pipe to break again

sfc-gh-asawicki commented 3 months ago

The fix was introduced in 0.89, not 0.88, so please provide the config and logs (run terraform with TF_LOG=DEBUG) on the 0.89+ version (I could not reproduce the issue given your config on the 0.89 version).

guillaumelecerf commented 2 months ago

We were having the same issue with:

file_format         = "FIELD_DELIMITER = '|' PARSE_HEADER = true"

Fix in https://github.com/Snowflake-Labs/terraform-provider-snowflake/pull/2885

sfc-gh-asawicki commented 2 months ago

The fix was included in v0.93.0. Please check it @guillaumelecerf @bmhah.

harukitake commented 2 months ago

Hey @sfc-gh-asawicki. I have the same issue on the version 0.93.0.

terraform configuration

resource "snowflake_stage" "stage" {
      database = "DATABASE_NAME"
      file_format = "TYPE = CSV NULL_IF = []"
      name = "NAME"
      schema = "SCHEMA_NAME"
      storage_integration = "INTEGRATION_NAME"
      url = "s3://aaaaa"
    }
sfc-gh-asawicki commented 2 months ago

I will take a look this week. Thanks for providing the config; it helps reproduce the issues.