huserben / TfsExtensions

Extensions for TFS 2015+ such as custom Widgets (require TFS 2017) and Build Tasks
MIT License
45 stars 22 forks source link

When I trigger another pipeline, the given parameter does not override the default parameter value #153

Closed gexiangge closed 3 years ago

gexiangge commented 3 years ago

I use below yaml to trigger another pipeline in the current pipeline. I explicitly pass the parameter and give the value "333", but when I check log of the triggered pipeline, the parameter still use the default value "222".

How can I use the value passed in to override the default value of that parameter? 160610-param

huserben commented 3 years ago

Hi @gexiangge

How is the parameter defined in the triggered build? It has to be a "variable" that can be overridden and not a runtime parameter: image

Runtime Parameters (https://docs.microsoft.com/en-us/azure/devops/pipelines/process/runtime-parameters?view=azure-devops&tabs=script) are not supported in this scenario by the task.

wbin666 commented 3 years ago

Hi @huserben
Is there a plan to support Runtime Parameters recently? It's very useful while expanding the templates conditionally.

I'm not familiar with js, but I'd like to contribute on this feature with your support (I don't like my current approach to trigger and monitor a pipeline via Azure DevOps API) If possible, I need your support to get started as a few points below:

1 How to setup a local development environment and run the project and the related test cases? I don't find the guide in Readme.md file.

2 As you're the master of this project, would you also like to share your view/analysis on this feature based on your current implementation?

Best regards, Bin Wang

wbin666 commented 3 years ago

Hi @huserben,

I see the related issue in microsoft backlog and you also commented on the issue. https://github.com/microsoft/azure-devops-node-api/issues/392

And I see the update in API 6.1 preview now. https://docs.microsoft.com/en-us/rest/api/azure/devops/build/builds/queue?view=azure-devops-rest-6.1

and the related discussion in this issue: https://github.com/huserben/TfsExtensions/issues/142

huserben commented 3 years ago

Hi @wbin666

First of all thank you for your offer and for making me aware that the runtime parameters are in the API preview at the moment.

As you noticed the task is currently using the devops-node-api, so as long as they don't add the feature to include the runtime parameters I think it's not really possible, unless the task would be rewritten and we would not depend on that api anymore (which is not something I would aim for).

The actual queuing happens in a library I was writing to abstract the details away and that is in its own git repo (earlier I did not use the devops-node-api so it was a bit more code in there :-)) --> Essentially the line that is triggering the build is this one: https://github.com/huserben/TfsRestService/blob/master/index.ts#L233

So once devops-node-api allows us to pass runtime parameters, we can rather easily pass it in there, the question would be what format they expect and how we get it from the users. The build parameters turned out to be quite messy because it's using a single textbox for defining multiple key value pairs. Easiest thing would probably be to go the same way, but I'm not sure if this is the best thing to proceed - how would you like to enter this data ideally?

Regarding your first question: Sorry that nothing is written in the readme - when I started I honestly didn't expect anyone to care or even contribute, so if you (and others) offer that it's really making my day.

Tasks are based on node.js, so you will need that installed if you haven't yet. Then essentially you can clone the repo, cd into the proper folder (there is a folder for every task in every major version - in this case it's probably about trigerbuildtaskv3) and do a npm install, that should get you all the dependencies.

After that you could build the js files via calling tsc. To run the tests you can do npm run test. The main entry point is the index.ts (this is what we define in the task.json that is describing the task and includes also the input a user can give).

I hope that's enough to get you started, please feel free to contact me in case you need more information

wbin666 commented 3 years ago

Hi @huserben,

Thanks for your kindly explanation. I get the point now. We have to wait for "devops-node-api" to support the latest Azure DevOps REST API (API Version: 6.1-preview).

BTW, I tried the rest api today and found both Build endpoint (6.1-preview.6) and Pipeline endpoint (6.1-preview.1) work well. and here are the workable samples.

Using Build Endpoint: $url = "https://dev.azure.com/{organization}/{project}/_apis/build/builds?api-version=6.1-preview.6" $parameters = '{"AgentPoolName":"' + $agentPoolName +'"}' $bodyString4Build = @{ definition = @{ id = $definitionId }; sourceBranch = $branchName ;parameters = $parameters; templateParameters=@{ agentNumberInPool = $agentNumberInPool; paraPipelineOwner = $paraPipelineOwner } } | ConvertTo-Json $buildresponse = Invoke-RestMethod -Method Post -ContentType application/json -Uri $url -Body $bodyString -Headers $DevOpsHeaders

The sample of $bodyString4Build is: image

Using Pipeline EndPoint: $url ="https://dev.azure.com/{organization}/{project}/_apis/pipelines/$definitionId/runs?api-version=6.1-preview.1" $bodyString4Pipeline = @{ variables = @{ AgentPoolName = @{value = $agentPoolName } }; templateParameters= @{agentNumberInPool = $agentNumberInPool; paraPipelineOwner = $paraPipelineOwner } } | ConvertTo-Json $buildresponse = Invoke-RestMethod -Method Post -ContentType application/json -Uri $url -Body $bodyString -Headers $DevOpsHeaders

The sample of $bodyString4Pipeline is: image

For TriggerBuild task, it would continue to use Build endpoints. Let's wait for devops-node-api.

huserben commented 3 years ago

Hi @wbin666

First of all sorry for the late answer, I somehow overlooked the email.

Good to know that the API seems to work correctly, I try to check out the node-api for when they support it and then try to include it in the task.

Thanks also for sharing the samples, so if others really need it they could use it as a template for a custom solution for the time being 👍

bergmeister commented 3 years ago

The node API has now been updated: https://github.com/microsoft/azure-devops-node-api/pull/426

huserben commented 3 years ago

Thanks @bergmeister for the hint :-)

I can look into defining the runtime parameters in a similar fashion like the build parameters work now. I'll keep you posted.

huserben commented 3 years ago

Hi @bergmeister @wbin666 @gexiangge

I just wanted to give you a quick update. I've just managed to run the task and queue a build and override a template paramter:

image image image

I will need a bit to test it more and properly update the task, I hope I can get it done during the next week (it depends a bit on my regular work though).

How is it implemented? Basically the same way as the existing parameters, so you can specify the runtime parameters within one big string: param1: MyFirstValue, param2: AnotherValue

Is this how you'd expected it to behave? Also could you maybe provide me some (simplified) examples of how you're gonna use it so I can test this as soon as possible to make sure it will actually work for your use cases?

As I have no experience with template parameters myself yet I'm glad for some sample yaml builds that I could run the task against with various inputs (and maybe some special cases if you can think of them and how it should behave in your view) :-)

I can't make any promises on managing to implement everything, but I'll sure gonna try and at least would be able to let you know what will and what will not work ;-)

Thanks for your assistance

gexiangge commented 3 years ago

@huserben when will you officially release this feature? I will give it a try? If I want to override both variable and parameter, can I just write them together in "buildParameters" section? like 'var1: value, param1: value'

wbin666 commented 3 years ago

Hi @huserben,

Glad to hear that the updated nodejs lib is ready.

My use case is to use both variables and parameters to trigger another pipeline with Personal Access Token, and wait for the triggered pipeline to finish with timeout setting (to make triggering pipeline failed if the triggered pipeline is cancelled, or failed, or timeout). I believe only template parameter is a new feature and others are existing functions that should still work well.

I also need an output variable to pass the url of triggered build to next task, so that I can define a following task "CancelTriggeredBuild" in triggering pipeline to cancel the triggered build automatically in case the triggering pipeline is cancelled or failed for any reason. I expect this existing function still works well.

Thank you.

huserben commented 3 years ago

@gexiangge I hope I can finish up this weekend, as the week was quite busy I didn't get to do as much as I would have liked. No there will be two distinct fields, as within the task I need to know whether it's a template parameter or a build variable. Technically I think there could be even a variable and a param with the same name...

@wbin666 Ok thanks for the info, I also fully expect that all still to work. In fact wait/cancel have not changed at all, only triggering. I'll let you know once the update is there.

huserben commented 3 years ago

Ok quick update from my side, I managed to run some tests today, and in general it looks good, but I saw a problem I have to investigate a bit more, so I would not feel confident releasing the new version already today.

Next work-days are a bit packed, so I'll keep you posted on the progress.

schmolfi commented 3 years ago

@huserben First of all, thank you very much for the great work.

Have you already deployed the new version? We currently need the new function for our project as soon as possible.

I really appreciate your work ... Thanks a lot

huserben commented 3 years ago

@schmolfi No the new version ain't in yet. I hope to get it done till Wednesday though.

huserben commented 3 years ago

@Everyone, the new version of the task is now availabe with support for template parameters.

Please let me know whether this solves your problems and whether it works as expected or not.

Thanks everyone for contributing, your help is appreciated!

schmolfi commented 3 years ago

AWESOME!!!! It's working, thank you so much!!

GitHubAre commented 3 years ago

@huserben Hello, I think you have done this change after 3.1.1 version, right? I dont see this behavior working on 3.1.1 which is available on MarketPlace. Do you plan to release 3.2.2 (I think this parameters feature is in 3.2.2)?

huserben commented 3 years ago

Hi @GitHubAre

hmm it should for sure be available in the latest version. Version 3.1.1 is the version of the extension, however the task inside has version 3.2.2. So if you install the latest version it should be avalable for you.

If you face a problem, e.g. something doesn't work as expected, I'd ask you to create a new issue with the details (e.g. your configuration, error message & logs).