Support for using Expressions within DataFactory Parameter objects

[LeighS]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Azure Data Factory supports using expressions within parameter objects. *An example ARM template that shows this (note the linkedServiceName.parameters.storageAccountEndpoint specifies a type of expression:

    "name": "ADLSBinary_IRA",
    "properties": {
        "linkedServiceName": {
            "referenceName": "GenericAzureDataLakeStorage_IRA",
            "type": "LinkedServiceReference",
            "parameters": {
                "StorageAccountEndpoint": {
                    "value": "@dataset().StorageAccountEndpoint",
                    "type": "Expression"
                }
            }
        },
        "parameters": {
            "StorageAccountEndpoint": {
                "type": "string"
            },
            "Directory": {
                "type": "string"
            },
            "FileSystem": {
                "type": "string"
            },
            "File": {
                "type": "string"
            }
        },
        "folder": {
            "name": "ADS Go Fast/Generic/IRA"
        },
        "annotations": [],
        "type": "Binary",
        "typeProperties": {
            "location": {
                "type": "AzureBlobFSLocation",
                "fileName": {
                    "value": "@dataset().File",
                    "type": "Expression"
                },
                "folderPath": {
                    "value": "@dataset().Directory",
                    "type": "Expression"
                },
                "fileSystem": {
                    "value": "@dataset().FileSystem",
                    "type": "Expression"
                }
            }
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"

Currently the linked_service.parameters seems to support string[map]interface{} but the schema is validating that element type is a string. This prevents us from using expressions in the parameters block

It would be great if we could specify these details as below:

resource "azurerm_data_factory_custom_dataset" "ADLSBinary" {
  name            = "ADLSBinary_IRA"
  data_factory_id = azurerm_data_factory.data_factory.id
  type            = "Json"
  linked_service {
    name = azurerm_data_factory_linked_custom_service.data_lake.name
    parameters = {
      StorageAccountEndpoint = {
        defaultValue = "@dataset().StorageAccountEndpoint"
        type = "Expression"
      }
    }
  }
  type_properties_json = <<JSON
{
            "location": {
                "type": "AzureBlobFSLocation",
                "fileName": {
                    "value": "@dataset().File",
                    "type": "Expression"
                },
                "folderPath": {
                    "value": "@dataset().Directory",
                    "type": "Expression"
                },
                "fileSystem": {
                    "value": "@dataset().FileSystem",
                    "type": "Expression"
                }
            }
        }
JSON
  description = "A Generic Binary File within Azure Data Lake"
  parameters = {
    StorageAccountEndpoint = ""
    Directory = ""
    FileSystem = ""
    File = ""
  }
}

New or Affected Resource(s)

• azurerm_data_factory_custom_dataset

Potential Terraform Configuration

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key.

References

https://docs.microsoft.com/en-us/azure/templates/microsoft.datafactory/2018-06-01/factories/datasets?tabs=bicep

• 13051

[CTCIevgenZasid]

The same issue.