Endpoint auth. parameter that is not confidential getting masked in agent logs

[GitHubSriramB]

2.122.0

Windows

VSTS

Attaching details about the custom endpoint type & logs that has this issue.

Endpoint details logged in agent:

{
  "data": {},
  "id": "cfa7e53f-5778-423c-af65-507a80224c79",
  "name": "cfa7e53f-5778-423c-af65-507a80224c79",
  "type": "habitatoriginendpoint",
  "url": "https://bldr.habitat.sh",
  "authorization": {
    "parameters": {
      "username": "********",
      "revision": "********",
      "publickey": "********",
      "password": "********",
      "githubauthtoken": "********",
      "useSudo": "********"
    },
    "scheme": "UsernamePassword"
  },
  "isReady": false
}

Endpoint contribution type:

{
  "id": "habitat-origin",
  "description": "Habitat Origin",
  "type": "ms.vss-endpoint.service-endpoint-type",
  "targets": ["ms.vss-endpoint.endpoint-types"],
  "properties": {
    "name": "habitatoriginendpoint",
    "displayName": "Habitat Origin",
    "url": {
      "displayName": "Habitat Depot URL",
      "helpText": "URL to the Habitat depot that will be used to deploy to"
    },
    "inputDescriptors": [],
    "authenticationSchemes": [
      {
        "type": "ms.vss-endpoint.endpoint-auth-scheme-basic",
        "inputDescriptors": [
          {
            "id": "username",
            "name": "Origin Name",
            "description": "Name of the Habitat origin",
            "inputMode": "textbox",
            "isConfidential": false,
            "validation": {
              "isRequired": true,
              "dataType": "string"
            }
          },
          {
            "id": "revision",
            "name": "Revision",
            "description": "Revision of the origin to use",
            "inputMode": "textbox",
            "isConfidential": false,
            "validation": {
              "isRequired": true,
              "dataType": "string"
            }
          },
          {
            "id": "publickey",
            "name": "Public Key",
            "description": "Public item of the origin key pair",
            "inputMode": "textarea",
            "isConfidential": false,
            "validation": {
              "isRequired": true,
              "dataType": "string"
            }
          },
          {
            "id": "password",
            "name": "Signing Key",
            "description": "Signing item of the origin key pair",
            "inputMode": "textarea",
            "isConfidential": true,
            "validation": {
              "isRequired": true,
              "dataType": "string"
            }
          },
          {
            "id": "githubauthtoken",
            "name": "GitHub Auth Token",
            "description": "Authentication token for GitHub for publishing Habitat packages.",
            "inputMode": "textbox",
            "isConfidential": "true",
            "validation": {
              "isRequired": true,
              "dataType": "string"
            }
          },
          {
            "id": "useSudo",
            "name": "Use Sudo",
            "description": "Use sudo on habitat commands",
            "inputMode": "combo",
            "isConfidential": false,
            "validation": {
              "isRequired": false,
              "dataType": "string"
            },
            "values": {
              "inputId": "useSudoValues",
              "defaultValue": "1",
              "possibleValues": [
                {
                  "value": "1",
                  "displayValue": "True"
                },
                {
                  "value": "0",
                  "displayValue": "False"
                }
              ]
            }
          }
        ]
      }
    ],
    "helpMarkDown": "Please provide the requested information so that Habitat packages can be built and deployed. The GitHub authentication token requires `user:email` and `read:org` scopes"
  }
}

[at-besa]

This issue is open for years and no one has yet figured out what's going on!? 🙄

In our pipelines, we are using Variable Groups connected to a Key Vault, right after the Download Secret task, all 1s are replaced with ***.

Have checked all the secrets in the Key Vault. None of them is 1. Of course, it could be part of a secret like something_1_something_else, but not by itself as a whole._

---

Hey everyone! We are introducing new agent knob: AGENT_IN_SECRET_LENGTH which will let you not to mask short secrets. You can specify values from 0 (all secrets will be masked as usual) to 4 (secrets shorten than 4 symbols will be ignored) Also, negative values will mean masking all the secrets as usual._

---

The knob is not working for the bug where that '1' is masked, we also checked our secrets and there is none of them having a 1. Also interesting fact is that it appears to happen only on the first stage, if the pipeline does have more stages the 1's are not masked except the first stage. The pipelines use the same agent for these stages, and the agent version is 2.217.2, which is the newest as by the time writing this.

So result is neither is the bug with suddenly masking 1 solved nor is there any workaround or solution to not have guessable logs where secrets are masked out of text which shouldn't be masked.

@KonstantinTyukalov @anatolybolshakov

[at-besa]

This issue is open for years and no one has yet figured out what's going on!? 🙄 In our pipelines, we are using Variable Groups connected to a Key Vault, right after the Download Secret task, all 1s are replaced with ***. Have checked all the secrets in the Key Vault. None of them is 1. Of course, it could be part of a secret like something_1_something_else, but not by itself as a whole._

Hey everyone! We are introducing new agent knob: AGENT_IN_SECRET_LENGTH which will let you not to mask short secrets. You can specify values from 0 (all secrets will be masked as usual) to 4 (secrets shorten than 4 symbols will be ignored) Also, negative values will mean masking all the secrets as usual._

The knob is not working for the bug where that '1' is masked, we also checked our secrets and there is none of them having a 1. Also interesting fact is that it appears to happen only on the first stage, if the pipeline does have more stages the 1's are not masked except the first stage. The pipelines use the same agent for these stages, and the agent version is 2.217.2, which is the newest as by the time writing this.

So result is neither is the bug with suddenly masking 1 solved nor is there any workaround or solution to not have guessable logs where secrets are masked out of text which shouldn't be masked.

@KonstantinTyukalov @anatolybolshakov

This also happens with the new Azure Agent with Version 3.217.1 as we just tried the pre-release version.

[KonstantinTyukalov]

@at-besa Do you have this or other extension in your pipeline? https://github.com/microsoft/azure-pipelines-agent/issues/1207#issue-257604223

Also, have you tried AZP_IGNORE_SECRETS_SHORTER_THAN knob name? Note that AGENT_MIN_SECRET_LENGTH knob name is not valid for now.

Also interesting fact is that it appears to happen only on the first stage, if the pipeline does have more stages the 1's are not masked except the first stage.

What tasks do you have in this first stage?

[StingyJack]

Also, have you tried AZP_IGNORE_SECRETS_SHORTER_THAN knob name? Note that AGENT_MIN_SECRET_LENGTH knob name is not valid for now.

@KonstantinTyukalov - this knob does not fix the problem that was reported. It actually makes the situation worse, thus invalidating it's intended purpose. Several users have told you this now. Please revert the knob and come up with a proper fix.

[at-besa]

@at-besa Do you have this or other extension in your pipeline? #1207 (comment)

Also, have you tried AZP_IGNORE_SECRETS_SHORTER_THAN knob name? Note that AGENT_MIN_SECRET_LENGTH knob name is not valid for now.

Also interesting fact is that it appears to happen only on the first stage, if the pipeline does have more stages the 1's are not masked except the first stage.

What tasks do you have in this first stage?

Yes i have tried the knob, the 1's are not masked anymore when i turn the knob value of AZP_IGNORE_SECRETS_SHORTER_THAN to 1, although this does not make any sense, as mentioned there is no secret which does even have this value. I agree with @StingyJack that this is just an workaround and not the solution to the actual problem.

regarding the tasks used in the first stage:

• checkout: self
• task: Bash@3
• task: Docker@2

The whole pipeline is running on an azure hosted vm with rocky as operating system.

as there is no secret stuff in this section i can share the yaml of this whole stage:

  - stage: Build
    jobs:
      - job: Build
        timeoutInMinutes: 10
        cancelTimeoutInMinutes: 2
        steps:
          - checkout: self
            fetchDepth: 2
            clean: true
            submodules: recursive

          ############ convert repo name to lowercase for openshift  ############
          - task: Bash@3
            displayName: 'Convert Repo to lowercase'
            inputs:
              targetType: 'inline'
              script: |
                LowerRepoName=$(echo $(Build.Repository.Name) | tr  [:upper:] [:lower:])
                echo "##vso[task.setvariable variable=RepoName]$LowerRepoName"

          ############ build image, tag and push it to registry  ############
          - task: Docker@2
            displayName: 'docker build and push'
            inputs:
              containerRegistry: 'openshift-registry'
              repository: '$(imageName)'
              command: 'buildAndPush'
              Dockerfile: '**/Dockerfile'
              tags: |
                $(Build.BuildNumber)
                latest

[at-besa]

any news on that ?

[github-actions[bot]]

This issue has had no activity in 180 days. Please comment if it is not actually stale

[at-besa]

180 days and still no comment of devs

[github-actions[bot]]

This issue has had no activity in 180 days. Please comment if it is not actually stale

[StingyJack]

Turn off this bot. It just makes the maintainers come across as "rude" instead of just "busy"

[github-actions[bot]]

This issue has had no activity in 180 days. Please comment if it is not actually stale

[StarWars999123]

Turn off this bot. It just makes the maintainers come across as "rude" instead of just "busy"

would consider them extremly ignorant now... This issue annoys me almost every day.