Snowflake-Labs / terraform-provider-snowflake

Terraform provider for managing Snowflake accounts
https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest
MIT License
552 stars 420 forks source link

[Bug]: snowflake_file_format resource ignores USE_LOGICAL_TYPE and REPLACE_INVALID_CHARACTERS #3185

Open a-schot opened 1 week ago

a-schot commented 1 week ago

Terraform CLI Version

1.9.8

Terraform Provider Version

0.96.0

Company Name

PGGM

Terraform Configuration

resource "snowflake_file_format" "parquet" {
  name     = "FF_PARQUET_DEFAULT"
  database = "db_name"
  schema   = "schema_name"

  format_type                = "PARQUET"
  compression                = "AUTO"
  binary_as_text             = true
  trim_space                 = false
  replace_invalid_characters = true
}

Category

category:resource

Object type(s)

resource:file_format

Expected Behavior

After running terraform apply running DESCRIBE FILE FORMAT in Snowflake shows:

An option in Terraform called use_logical_type with the same default as the Snowflake which sets (and checks when planning) the USE_LOGICAL_TYPE property for the relevant file formats.

Additionally, running

ALTER FILE FORMAT db_name.schema_name.ff_parquet_default
SET 
USE_LOGICAL_TYPE = TRUE
REPLACE_INVALID_CHARACTERS = TRUE
;

And rerunning terraform apply does not detect changes in the configuration, leading me to suspect it's not being checked in the provider.

Actual Behavior

After running terraform apply running DESCRIBE FILE FORMAT in Snowflake shows:

The defaults for both these properties

Steps to Reproduce

  1. Apply the terraform configuration
  2. Check the resulting file format using DESCRIBE FILE FORMAT db_name.schema_name.ff_parquet_default
  3. Run again (my result: no changes)
  4. Alter the file format using the snippet in Actual Behavior
  5. Run again (my result: no changes)

How much impact is this issue causing?

Low

Logs

No response

Additional Information

No response

Would you like to implement a fix?

sfc-gh-jmichalak commented 1 week ago

Hi @a-schot 👋

I think this is similar to https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3163, but that issue missed USE_LOGICAL_TYPE. Our answer is similar here: we are currently focused on pushing all necessary things toward the recently announced v1. We'll fix this during rework of file_format resource after v1. For now, please try to utilize the unsafe_execute as a way to alter or completely manage the file format.