Property row_delimiter does not resolve to correct default value when omitted

[tskrgulja]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.5.2

AzureRM Provider Version

3.64.0

Affected Resource(s)/Data Source(s)

azurerm_data_factory_dataset_delimited_text

Terraform Configuration Files

# This config hasn't been applied
provider azurerm {
  version = "3.64.0"
  features {}
}

resource azurerm_resource_group example {
  name     = "example"
  location = "uksouth"
}

resource azurerm_storage_account example {
  name                      = "example"
  resource_group_name       = azurerm_resource_group.example.name
  location                  = azurerm_resource_group.example.location
  account_kind              = "StorageV2"
  account_tier              = "Standard"
  account_replication_type  = "GRS"
  enable_https_traffic_only = true
}

resource azurerm_storage_container example {
  name                  = "example"
  storage_account_name  = azurerm_storage_account.example.name
  container_access_type = "private"
}

resource azurerm_data_factory example {
  name                = "example"
  resource_group_name = azurerm_resource_group.example.name
  location            = azurerm_resource_group.example.location
}

resource azurerm_data_factory_linked_service_azure_blob_storage example {
  name                     = "example"
  resource_group_name      = azurerm_resource_group.example.name
  data_factory_name        = azurerm_data_factory.example.name
  integration_runtime_name = "AutoResolveIntegrationRuntime"
  connection_string        = azurerm_storage_account.example.primary_connection_string
}

resource azurerm_data_factory_dataset_delimited_text example {
  name                = "example"
  resource_group_name = azurerm_resource_group.example.name
  data_factory_name   = azurerm_data_factory.example.name
  linked_service_name = azurerm_data_factory_linked_service_azure_blob_storage.example.name
  parameters          = {
    "subfolder" = "" # Map value sets parameter default value
  }

  first_row_as_header = true

  schema_column {
    name = "example"
    type = "String"
  }

  azure_blob_storage_location {
    container            = azurerm_storage_container.example.name
    path                 = "@dataset().subfolder"
    dynamic_path_enabled = true
  }
}

Debug Output/Panic Output

N/A

Expected Behaviour

Please see issue #13545. It is Resolved but row_delimiter property has not been fixed.

Actual Behaviour

Please see issue #13545. It is Resolved but row_delimiter property has not been fixed.

Steps to Reproduce

No response

Important Factoids

No response

References

Please see issue #13545. It is Resolved but row_delimiter property has not been fixed.

[fdmsantos]

Any news?

How are you setting the default value (without omitting the row_delimiter) ? I tried several ways, but without any success.

[tskrgulja]

Hi,

As a workaround I have created an extra azapi resource which sets Row delimiter to default value. I have wrapped it inside the module to make it easily reusable (but you can use code directly without wrapping if you want).

Something like this (your versions etc. might be different):

Module contents (set-default-row-delimiter/main.tf):

terraform {
  required_providers {
    azapi = {
      source  = "Azure/azapi"
      version = "=1.7.0"
    }
  }
}

variable "resource_id" {
  type = string
}

resource "azapi_update_resource" "this" {
  type        = "Microsoft.DataFactory/factories/datasets@2018-06-01"
  resource_id = var.resource_id

  body = jsonencode({
    "properties" : {
      "typeProperties" : {
        "rowDelimiter" : null
      }
    }
  })
}

Example of usage (be sure to set correct path to the module):

resource "azurerm_data_factory_dataset_delimited_text" "example" {
  name                = "exampleDataset"

  # rest of the parameters are omitted for brevity
}

module "set_default_row_delimiter_source" {
  source = "../set-default-row-delimiter"

  resource_id = azurerm_data_factory_dataset_delimited_text.example.id
}

Hope this helps :)

[aurbor]

@tskrgulja - this looks incredibly helpful!

We've just resorted to manually setting the value manually after deployment as we couldn't come up with a reasonable workaround. This looks really positive.

Will test it in one of our future releases and see if it does the trick for us. Really appreciate you sharing this! Thank you!

Evan

[aurbor]

I've just fired this up in a dev environment and it looks great. I've adjusted it to loop through my delimited datasets as we're creating a few as part of a loop that parses JSON files.

Delimited Dataset resource:

resource "azurerm_data_factory_dataset_delimited_text" "adfdsdt" {  
  for_each = { for x in local.adf_datasets : x.filename => x.config if x.config.properties.type == "DelimitedText" }

  name                = each.value.name
  # Other properties removed
}

Module running right below the above resource block:

# Set the default row delimiter for each delimited dataset
module "set_default_row_delimiter" {
  for_each = azurerm_data_factory_dataset_delimited_text.adfdsdt

  source = "./set-default-row-delimiter"
  resource_id = each.value.id

  depends_on = [ azurerm_data_factory_dataset_delimited_text.adfdsdt ]
}

Great work @tskrgulja - appreciate it again!

[fdmsantos]

Tks @tskrgulja for sharing your workaround.