how to pass a list of double quoted strings as an azure pipeline variable input to terraform task?

[mathurk1]

Hi Team,

I am trying to pass a list of strings as an azure pipeline variable to a terraform task.

I have a variable created for the pipeline called such that ips = [\"1.1.1.1\", \"2.2.2.2\"] and I can access this in the azure pipeline using $(ips)

I want to pass this variable as mentioned here in the official terraform documentation: terraform apply -var='image_id_list=["ami-abc123","ami-def456"]' to the terraform plan task

here is my terraform plan section of the yaml:

        - task: TerraformTaskV1@0
          inputs:
            provider: 'azurerm'
            command: 'plan'
            commandOptions: '--var-file variables/$(Build.SourceBranchName)/$(Build.SourceBranchName).tfvars  -var=''allowips=$(ips)''' 
            environmentServiceNameAzureRM: $(linked_service)
          displayName: 'Terraform plan'

when the pipeline runs, it fails at the terraform plan step with the error -

terraform plan --var-file variables/dev/dev.tfvars -var='allowips=[ \1.1.1.1", "2.2.2.2" ]'
stat \1.1.1.1", "2.2.2.2" ]': no such file or directory
##[error]Error: The process '/opt/hostedtoolcache/terraform/0.12.28/x64/terraform' failed with exit code 1

I have also tried various combinations of for declaring the azure pipeline variable like: ips = ["\"1.1.1.1\", \"2.2.2.2"\]

which also gives the same error -

terraform plan --var-file variables/dev/dev.tfvars -var='allowips=[ "1.1.1.1", "2.2.2.2" ]'
stat "1.1.1.1", "2.2.2.2" ]': no such file or directory
##[error]Error: The process '/opt/hostedtoolcache/terraform/0.12.28/x64/terraform' failed with exit code 1

I have tried to a lot of combinations of setting quotes or double quotes on the commandOptions value and also the ips variable but I am getting the same errors (or valid terraform errors).

It seems the first double quote in the azure variable seems to causing some issues. Any pointers on how to pass a list of double quoted strings as an azure pipeline variable input to terraform task is much appreciated. Thanks!

edit:

I tried to hardcoded the values directly in the yaml but still getting errors.

commandOptions: '--var-file variables/$(Build.SourceBranchName)/$(Build.SourceBranchName).tfvars -var=''allowips=[\"1.1.1.1\", \"2.2.2.2\"]'''

and it basically does not accept the ip values and asks for a manual entry:

/opt/hostedtoolcache/terraform/0.12.28/x64/terraform plan --var-file variables/dev/dev.tfvars -var='allowips=[\1.1.1.1", "2.2.2.2"]'
var.allowips
  Enter a value: 

again it seems there is a problem around the first double quote.

[AmrutaKawade]

I am not able reproduce this issue. it getting passed like this for me C:\hostedtoolcache\windows\terraform\0.12.3\x64\terraform.exe plan --var-file variables/dev/dev.tfvars -var='allowips=[ 1.1.1.1, 2.2.2.2 ]'

[mathurk1]

perhaps because you are using a windows agent and I am using ubuntu?

[CaptainStealthy]

@mathurk1 Have you tried using multi-line syntax for passing the command options? I've found this makes code a lot more readable, and it reduces the complexity a lot for situations like this 😆

# This is pseudocode (missing stages/jobs/other tasks, etc), but you get the point

variables:
  ips: '["1.1.1.1", "2.2.2.2"]' # This could be defined in a var group as well

steps:
  - task: TerraformTaskV1@0
    inputs:
      provider: azurerm
      command: plan
      # The >- replaces the newlines with spaces and strips any newline from the end
      # Similar to the "|" multi-line syntax for Bash/PowerShell scripts
      # More info: https://yaml-multiline.info/
      commandOptions: >-
        --var-file variables/$(Build.SourceBranchName)/$(Build.SourceBranchName).tfvars
        -var='allowips=$(ips)'
      environmentServiceNameAzureRM: $(linked_service)
    displayName: Terraform plan

The other thing you could do is declare an empty variable in a .tfvars file and replace/inject the value using Bash or PowerShell in the pipeline, right before the Terraform tasks run.

[mathurk1]

Thanks @CaptainStealthy, definitely makes the code more readable. However it does not resolve the issue I am facing.

The issue seems to be in how the TerraformTaskV1@0 parses double quotes in the commandOptions arguments.

• it strips all double quotes (as you can see in @AmrutaKawade's output)
• if the double quotes are preceded by an escape character like ips = [\"1.1.1.1\", \"2.2.2.2\"] then the terrafrom task strips the first double quote. In this case looks, the output looks like: terraform plan --var-file variables/dev/dev.tfvars -var='allowips=[ \1.1.1.1", "2.2.2.2" ]'

[CaptainStealthy]

@mathurk1 Blegh... I was hoping my solution would solve the issue, lol. I haven't actually tried passing -var= command options like this in pipelines, so I wish I could be of more help.

Is there a way you can put that variable into a .tfvars file instead? If the values need to be decided by the pipeline, you could keep the value in the tfvars file empty, and just before the TF steps, use Bash or PowerShell to inject your pipeline variables into the file. There's token replacement extensions that can do this for you too.

[mathurk1]

Thanks for the suggestion @CaptainStealthy. If the values are directly injected in the tfvars file directly, I am sure that would work since we will bypass the need to pass double quoted strings.

[20shivangi]

@mathurk1 Are you still failing the issue ?

[mathurk1]

Hi @20shivangi - yes, I am still facing the issue...

[20shivangi]

@mathurk1 When you are injecting the values are directly in the tfvars file, even then you are facing the issue ?

[mathurk1]

the injection is a work around that we have implemented. I am not able to use the plan command as is though.

[20shivangi]

It is great that workaround is working for you. We will track this issue to ensure plan command works.

[ichhaporia]

How is this an "enhancement" and not a bug? It is 2023 and the issue still persists.

[aajetunde]

If you're still struggling with escaping quotes, this is what the documentation says:

If your arguments contain double quotes ("), escape them with a slash (\), and surround the escaped string with double quotes (").

https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/npm-v1?view=azure-pipelines

I was having exactly the same issues until I escaped it properly e.g. "sample" was escaped as "\""sample"\""

Hope this helps