Use Variables to specify SecureFile to download in DownloadSecureFile Task

[fume]

Hi, this is not an issue but just a suggestion about the DownloadSecureFile task.

I think that could be very useful to be able to specify the SecureFile name via a variable, but the "secureFile" type only allows the selection via UI. This will also allow to create parametrized Task groups containing this task.

thanks, regards

[pwil301]

Completely agree with this request. Not being able to use a variable or dynamically select a secure file prevents using the library for some purposes it is otherwise perfectly suited for.

As mentioned here (quote below in case the link goes stale): https://stackoverflow.com/questions/49678831/how-to-use-variable-to-specify-secure-file-input-value-in-tfs-2017u2-build-and-r

"Our workflows include generating new x509 certificates and pushing them into the Secure Files library and setting properties on the entries such as CN, thumbprint, SAN, etc. We then need to pull the file down to an agent for tasks later in the workflow. However, because the file is not in the library at design time, we cannot select it in the list.

We tried implementing a custom download task. However, if you don't use the "securefile" input type and specify the exact file you need when defining the build, TFS won't issue a download ticket to the agent and you thus can't download the actual file (only metadata)."

Any chance for a solution?

[fume]

is there any ETA for this enhancement?

thanks, regards

[DuncanMillard]

+1 request for this

[ericsciple]

This is something we want to enable.

The closest fit today would be conditional tasks or yaml templates. Both requiring knowing the file up front.

[SqlSusan]

Also waiting for this enhancement. Is there a workaround?

[madhurig]

This is supported via yaml but we don't have an ETA on support in the classic designer.

E.g.

[EdwardBlair]

@madhurig This doesn't work if you nest this within a job definition

[fume]

@madhurig still no ETA for the classic designer?

thanks, regards

[johncrim]

This works today - you can use the step name: to provide a secure file variable prefix. Then add .secureFilePath to the prefix for the variable name. The following example works:

- task: DownloadSecureFile@1
  name: CaCert
  displayName: 'Download CA cert'
  inputs:
    secureFile: 'ca.pem'

- script: |
    echo Installing $(CaCert.secureFilePath) to trusted CA directory...
    sudo chown root:root $(CaCert.secureFilePath)
    sudo chmod a+r $(CaCert.secureFilePath)
    sudo ln -s -t /etc/ssl/certs/ $(CaCert.secureFilePath)

[fume]

Hi @johncrim, this works only for the YAML designer (as stated by @madhurig in April). I was interested in the ETA for the feature to be available in the classic designer.

thanks, regards

[johncrim]

@fume - sorry for the mixup. The original issue says you can set it in the UI, which is true; and you can set it in YAML, though it wasn't documented.

I suppose I don't know what the classic design is....

[fume]

@johncrim UI = Classic Designer and you CAN'T set the file via a variable. Still waiting for an ETA :)

regards, F.

[henning-krause]

Since we currently cannot use YAML do define Release Pipelines with the On-Prev Azure DevOps Server, we need this in the classic UI.

[joe-walker-msft]

I completely agree with @henning-krause - The lack of ability to set this in the Release pipeline is frustrating.

+1 request for this feature!

[efunkenbusch]

This is supported via yaml but we don't have an ETA on support in the classic designer.

E.g.

This doesn't seem to work at all, I have to specify both the certFileName and certPwd specifically, or i get an error (in the case of the certFileName it's a validation error, saying the job isn't valid) and in the case of the certPwd it clearly is including the literal "$(p12Pass)" name in the command line rather than replacing with the variable.

Everything builds correctly if I specify them as literals, but not when using variables at all. This is for the InstallAppleCertificate@2 task, which looks almost exactly like the image you show.

EDIT:

I think the above will work if your password is just an instance variable, but if your password is a group variable that you link (which it should be, in order to secure the password) then you have to do some gymnastics.

You have to redefine the group variable as a local variable, essentially.. redeclare actually, not really redefine using the alternate syntax.

The reason for this seems to be that the task cannot access the group library variables directly, so you have to locally declare them at the top level.

variables:
- group: 'my-variables'
 - name: 'p12FileName2'
   value: '$(p12FileName)'
 - name: 'p12Password2'
   value: '$(p12Password)'

steps:
 -task: InstallAppleCertificate@2
  inputs:
   certSecureFile: '$(p12FileName2)'
   certPwd: '$(p12Password2)'
   keychain: 'temp'
   deleteCert: true

[PadmanavAgarwal]

Hi, I am using a secureFile in my pipeline yaml file as below :

inputs:
   secureFile: '$(SECURE_FILE)'

However, this works only if I use a global variable. If the 'SECURE_FILE' variable is defined in stage or job scope, the variable is not fetched. Please note that this works fine for all other variables, and the issue is only with secureFile.

Is this expected ? What is the specific issue with Stage/Job scope variables and any workaround for the same ?

[dearsi-mocha]

This does not work for me either. I want to retrieve the name of the secure file dynamically (output from another task)

Job Deploy_Service_DEV_dev_eastus_Log: Step sslCertificate input secureFile references secure file $(certificateName).pfx which could not be found.

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[Rickvanderwaal]

This issue is still relevant. I want this feature to pass the name of a secure file to a task group in my pipeline.

[connor-ricks]

Similar to what I just made #13783

Our source code builds 4 applications and we want to a variable to inject the correct certificate. However, this doesn't appear to work.

[LaurentGoderre]

It appears to me that it is failing because the validation is trying to validate the secure files before variable substitution happen and gives an error saying the secure file doesn't exist.

[nielslbeck]

If I specify the name of the secure file to eg. InstallAppleProvisioningProfile@1 from a variable defined at eg. the stage level it doesn't work. If I specify it globally there's no problem.

So I agree. This should be fixed.

[andmig-ilty]

Any update on this? I just had the same issue (YAML pipeline) and found this github issue... Two stages that need to download different secure files, receive the secure file name via stage-scoped variables... Nada... It only works if the variable is globally defined. That's definitively a wrong behavior.

[pjivers-bom]

+1

[anatolybolshakov]

Hi everyone, this looks like an enhancement for Azure DevOps itself, this repo is mostly for pipeline tasks - could you please fill in ticket in https://developercommunity.visualstudio.com to get right eyes on it?

[frankfuu]

@anatolybolshakov , i think this idea has already been "rejected" https://developercommunity.visualstudio.com/t/installsshkey-task-cant-resolve-variable-reference/984321

It's kind of sad really. Imagine a company that needs to build different versions of an app based on a branch/brand/variant etc, that relies on using a different certificate (secure file) running from the same code base.

[connor-ricks]

@frankfuu we do exactly that in my current job. We build 4 applications all from the same codebase. They all utilize different certificates, and they all go to different App Store connects. This problem is solvable in Azure's current state.

Checkout the comments in #13783

[frankfuu]

Thanks @Zanchee , I'll give it a go.

[skelso01]

I have a similar problem. In my case the securefile is created in the first stage and used in the second stage. If the secure file doesn't exist before the pipeline run, the pipeline does not start. So I then created an empty file and uploaded it. But since I can't update the secure file via REST API, I have to delete it first and upload the created one. So the secure file gets a new ID and the second stage doesn't start at all. It fails without any error message. If I remove the download secure file task from the second stage everything works. I assume that the ID of the secure file is stored in the background during the pipeline start, which is why the second stage can't find the secure file cause of the new ID.

Therefore my question, whether there is already an update for this issue? Or when a solution release is planed for this.

[sandorfr]

I ran into this problem 4 years ago, sad to see it has not progressed since then...

It's funny how it's seen as an enhancement when literally it's just a bug. It's the only place we can't use variables from matrixes.

[sumo300]

We've run into this exact issue. We need to be able to use environment-specific variables, which are in the stage scope. Please fix this. This is a pretty significant defect.

[fume]

even if late, i would like to say happy birthday to this issue which is still open and now is 4 years old!

Happy Birthday #6885 !!!

[twist250]

OMG, in the last two days I thought I'm doing it wrong, please fix this issue

[the-rule]

Have also spent quite a lot of time until realized it might not be supported at all! This is a very essential feature for automation and templating, @leantk could you please prioritize it?

[grofit]

I think I have a variant of this issue where we want to put some of our common stages into a template yml and then reference that for stages with different params, i.e deploy-to-env.yml which azure-pipelines.yml references.

The problem we have is that the moment we put the download secure file into the template step rather than the root pipeline steps it just refuses to download it and shows us the $(VARIABLE_GROUP_VAR_NAME) in the error as if its trying to process that secure file before its done the runtime substitution, which I think someone else said up top.

[A154858]

+1 Same problem with us. Could not use variable as secure file name. No matter what I try.

[mdeery]

+1

[themaroqa]

Just found an easy workaround for this issue that might partially fix the problem on the classic UI, specifically it may helps on having parametrized taskgroup containing this task.

• Create a taskgroup containing the DownloadSecureFileV1 task (you can obviously add as many task as you want).
• Manually edit the json so that you can pass to the DownloadSecureFileV1 task the variable as below

  "inputs":{
          "secureFile":"$(secureFile)",
          "retryCount":"3",
          "socketTimeout":""
       },

• Then define a new input as below

  {
      "aliases":[],
      "options":{},
      "properties":{},
      "name":"secureFile",
      "label":"secureFile",
      "defaultValue":"",
      "required":true,
      "type":"secureFile", # or "type":"string",
      "helpMarkDown":"",
      "groupName":""
   }

The UI won't help as it still won't show the variable in the taskgroup, it eventually show it in the release if the input type is secureFile. No way to have evidence in the UI of the file name if the input type is string. Anyway while running the release it succesfully download the file (tested adding a task that print the content of the file)

It is as dirty as it seems.

[bandarlogen]

The solution described below may help somebody get around the problem.

You can use parameters instead of variables.

steps:
  - template: template.yml
    parameters:
      parameter_for_secure_file: 'example'

And in template.yml:

parameters:
  parameter_for_secure_file: ''

steps:
  - task: DownloadSecureFile@1
    name: testSecureFile
    displayName: 'Download Secure File'
    inputs:
      secureFile: 'prefix-${{ parameters.parameter_for_secure_file }}-postfix'

[amarkevich]

Use compile time template expression ${{ variables.var }} instead of runtime macro $(var):

  variables:
    - name: file_prefix
      value: '${{ parameters.deploy_env }}'
...
        - task: DownloadSecureFile@1
          name: truststore_file
          inputs:
            secureFile: "${{ variables.file_prefix }}.truststore.jks"

[fume]

@bandarlogen , @amarkevich this could work for YAML pipelines but not for Classic UI Pipelines

[hareesh309]

Looking for this feature to use variable for downloading secure file in Classic(UI) pipelines, instead of selecting from drop down for every release..

[EdwardBlair]

Amazing. Very impressive

On Thu, 4 Aug 2022 at 14:51, Matteo Marenghi @.***> wrote:

Just found an easy workaround for this issue that might partially fix the problem on the classic UI, specifically it may helps on having parametrized taskgroup containing this task.

• Create a taskgroup containing the DownloadSecureFileV1 task (you can obviously add as many task as you want).
• Manually edit the json so that you can pass to the DownloadSecureFileV1 task the variable as below

"inputs":{ "secureFile":"$(secureFile)", "retryCount":"3", "socketTimeout":"" },

• Then define a new input as below

{ "aliases":[], "options":{}, "properties":{}, "name":"secureFile", "label":"secureFile", "defaultValue":"", "required":true, "type":"secureFile", # or "type":"string", "helpMarkDown":"", "groupName":"" }

The UI won't help as it still won't show the variable in the taskgroup, it eventually show it in the release if the input type is secureFile. No way to have evidence in the UI of the file name if the input type is string. Anyway while running the release it succesfully download the file (tested adding a task that print the content of the file)

It is as dirty as it seems.

— Reply to this email directly, view it on GitHub https://github.com/microsoft/azure-pipelines-tasks/issues/6885#issuecomment-1205287101, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB2QOB7EHGUAKGQ6IJTI7DVXPDELANCNFSM4EZAONKQ . You are receiving this because you commented.Message ID: @.***>

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[sandorfr]

This issue is not stale by any means.... It's still relevant...

[fume]

no stale even after 5 years! 😄

[snicro]

Definitely not stale. We need this!

[cd83]

Absolutely still needed! @fume time to open another 5 year long request?

[b-zijlstra]

Just ran into this issue. Would really appreciate if this gets fixed.

[nuzolx]

Same issue for us. Using a variable group for secure file name.