Support for azurerm_data_factory_pipeline to allow for parameters and variables of non string types

[johndvnguyen]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Datafactory pipelines created via terraform have the optional arguments of parameters and variables. These arguments only accept string maps of the parameter/variable names at this time.

It would great if these accepted parameter/variable blocks that allowed types similar to how global variables for the datafactory are created

New or Affected Resource(s)

• azurerm_data_factory_pipeline

Potential Terraform Configuration

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key.

References

• 0000

[Alfio-Alberto-Pellegrino]

hello, I need to deploy a boolean variable type, in the arm template the code is:

"variables": { "END": { "type": "Boolean", "defaultValue": false } }

do you have some update?

[rbollet]

Hello I have same problem, I need to deploy an array variable type. Do you have some update ?

[embarrassedmilk]

As far as I understand, it was implemented like this due to the limitations in Terraform SDK (correct me if I'm wrong). Now I'm curious what was the reason to choose a map of strings instead of accepting plain json (like in activities_json).

If you guys would be open for a PR that adds variables_json, I could give it a go

[gregops312]

I do believe I looked at this lat week and saw that the AzureSDK supports other options, which is what the provider is using under the hood. If you follow the trace

• Expansion of parameters & variables
• Looking at variable expansion a little deeper finds you
• Even deeper, if you inspect the above of VariableSpecification

// VariableSpecification definition of a single variable for a Pipeline.
type VariableSpecification struct {
    // Type - Variable type. Possible values include: 'VariableTypeString', 'VariableTypeBool', 'VariableTypeArray'
    Type VariableType `json:"type,omitempty"`
    // DefaultValue - Default value of variable.
    DefaultValue interface{} `json:"defaultValue,omitempty"`
}

And if you then circle back to the second bullet point you'll see that the provider code only ever uses VariableTypeString. I want to obviously get in there and open a PR, I just haven't found the time.

[embarrassedmilk]

Indeed, AzureSDK does support it, saw this code as well. Where I think it might get tricky is here. In ideal world i'd like to be able to do something like this:

variables = {
  var1 = "bar"
  var2 = false
  var3 = [1,2,3]
}

Also found this issue suggesting that it's not possible at the moment.

[gregops312]

Ya, I would suspect you'd have to just replace this with a validation function like [here[(https://github.com/hashicorp/terraform-provider-azurerm/blob/d1f34cbc1754e4c29f9ab894bca1a89d431f0cfc/internal/services/datafactory/data_factory_pipeline_resource.go#L98)

I'm not sure about the issue, I would have to dig into that.

I would also be fine with something more complex like, but I think the more implicit the better in this case since that is the terraform way by default.

variables = {
  var1 = {
    type = string
    default = "point"  
  }
}

[embarrassedmilk]

Hmm, not sure if simple replacing will work. From the Schema documentation it looks like it needs map elements to be one of the primitive types, so nothing complex is accepted. The only type-safe workaround I can think of is to do something like this which is also not ideal (in my opinion). Or we can turn variables into a json string, deserialize it and pass to the pipeline after it's validated. Something similar to iam policy in aws provider. Usage would look like this:

variables = jsonencode({
  var1 = {
    type = string
    default = "point"  
  })
}

[luisamador]

I've just come across this issue too and it's preventing my team from using Terraform for managing our Azure Data Factory instances :(

[paradox2707]

Please, fix this issue, I have needed pipeline variable with Array type and I had to find a workaround for it. In my case it was possible, but next time I doubt it will be affordable

[abratv]

https://github.com/hashicorp/terraform-provider-azurerm/issues/13131

[JuupHietbrink]

https://github.com/hashicorp/terraform-provider-azurerm/issues/13051