microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.46k stars 2.6k forks source link

[BUG]: FileTransform@1 broken when trying to replace keys starting with `input* in JSON #19172

Closed ProtossGP32 closed 2 months ago

ProtossGP32 commented 10 months ago

Task name

FileTransform@1

Task version

1

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

ubuntu-latest

Task log

Starting: File transformation: test_file.json
==============================================================================
Task         : File transform
Description  : Replace tokens with variable values in XML or JSON configuration files
Version      : 1.220.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/file-transform
==============================================================================
Applying JSON variable substitution for **/test_file.json
Applying JSON variable substitution for /home/vsts/work/1/s/templates/files/test_file.json
Substituting value on key a with (string) value: This is the value for a
Substituting value on key output with (string) value: b_2 output string
Substituting value on key random with (string) value: b_2 random string
Substituting value on key output_path with (string) value: This is the value for output_path
Substituting value on key input_path with (string) value: undefined # This is correctly defined as a variables and instead appears as undefined, thus removing the key from the source JSON
JSON variable substitution applied successfully.
Finishing: File transformation: test_file.json

Relevant log output

There are no errors in the log but currently defined variables that start with `input_*` are incorrectly assigned:

Substituting value on key input_path with (string) value: undefined

Aditional info

Current configuration of the task is as follows:

stages:
  - stage: FileTransformStage
    variables:
      - template: 'templates/variables/test-file-values.yaml'
    jobs:
      - job: FileTransformJob
        steps:
          - checkout: self
            submodules: true
          # Replace variables in `test_file.json`
          - task: FileTransform@1
            displayName: "File transformation: test_file.json"
            inputs:
              folderPath: '$(Build.SourcesDirectory)/templates/files/'
              fileType: 'json'
              targetFiles: '**/test_file.json'
          # Output transformed file
          - task: Bash@3
            displayName: "Check File Transformations"
            inputs:
              targetType: 'inline'
              workingDirectory: '$(Build.SourceDirectory)'
              script: |
                echo "Showing content of $(Build.SourcesDirectory)/templates/files/test_file.json"
                cat $(Build.SourcesDirectory)/templates/files/test_file.json
                echo ""
                echo "==========================="
                echo "Checking template variables:"
                echo "input_path: $(input_path)"

test_file.json is as follows:

{
  "a": "DUMMY",
  "b": {
    "b_2": {
      "input": "DUMMY",
      "output": "DUMMY",
      "random": "DUMMY"
    },
    "b_3": {
      "input": "DUMMY",
      "output": "DUMMY",
      "random": "DUMMY"
    },
    "b_4": {
      "input": "DUMMY",
      "output": "DUMMY",
      "random": "DUMMY"
    }
  },
  "output_path": "DUMMY",
  "input_path": "DUMMY"
}

test-file-values.yaml is as follows:

variables:
  a: "This is the value for a"
  output_path: "This is the value for output_path"
  input_path: "This is the value for input_path"
  b.b_2.output: "b_2 output string"
  b.b_2.random: "b_2 random string"

When looking at the transformed file, the next bash task outputs the following:

Showing content of /home/vsts/work/1/s/templates/files/test_file.json
{
    "a": "This is the value for a",
    "b": {
        "b_2": {
            "input": "DUMMY",
            "output": "b_2 output string",
            "random": "b_2 random string"
        },
        "b_3": {
            "input": "DUMMY",
            "output": "DUMMY",
            "random": "DUMMY"
        },
        "b_4": {
            "input": "DUMMY",
            "output": "DUMMY",
            "random": "DUMMY"
        }
    },
    "output_path": "This is the value for output_path"
}
===========================
Checking template variables:
input_path: This is the value for input_path

As you can see, all defined variable keys are replaced except the ones that start with input_*, which are completely wiped from the original JSON file as FileTransform thinks that its value is undefined.

ProtossGP32 commented 8 months ago

Hi! Any news on this bug? This right now is a dealbreaker for all of our CI pipelines

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days