Triggered builds building specific folders within a shelveset

[jfinnegan3]

Hello again @huserben!

Thanks again for all the great work you've done! I have another enhancement that would help me in my current situation and perhaps others as well.

I would like to trigger a build using only specific folders within a shelveset. This way, I can user the GetSources step to get specific sources, and then only unshelve the parts of the shelveset that would be relevant to those sources. If I use the whole shelveset, then I receive folder mapping errors on the unshelving step.

The TFS command on the unshelving step is: tf unshelve [/move] [shelvesetname[;username]] itemspec where itemspec can be the folder from which you want the items to be unshelved. This is called at the conclusion of the GetSources step.

I think this should be able to be done with an unchecked "Use same source branch as triggered build", but I tried it and it doesn't work. This might be again because the shelveset isn't escaped, and, as was my previous issue, I need shelveset names escaped or I get JSON parsing errors. I had that field formatted as such: shelveset_name folder_path I tried quotations and other things, and it didn't work

I would also think the "Define additional parameters..." should work when I have a variable that is Build.SourceBranchName or Build.SourceBranch. Where I again specified it as: Build.SourceBranchName: shelveset_name folder_path, ... But this also didn't yield any positive results.

Would it be possible for you to look into this for me?

Thanks!

[huserben]

Hi @jfinnegan3

Sorry I didn't really get it, what exactly are you trying to achieve? Is the problem that the Shelveset name is not escaped correctly when passed to the triggered build itself? Or you want to use a variable in the GetSources step in the beginning?

Maybe you can explain it for dummies so I can understand your idea :-)

[jfinnegan3]

Not a problem @huserben,

Perhaps, looking back, I wasn't as clear as I could have been. First, the escaping is not an issue. I've looked into it and that field is escaped and displays correctly before the API call.

I'll break it down more: This all starts with a Gated Check-in. A dev goes to check their code in. The Gated Check-in triggers 2 builds, one for each of two distinct parts of the codebase. One build that is triggered only gets sources for a part of the project (this can be specified within the Get Sources task). I would then like that build to unshelve only part of the shelveset generated for the overall check-in (the changes corresponding to the folder(s) from which the sources were gotten). Because, as I wrote above, if I was to unshelve the whole shelveset after only using the Get Sources task for a subset of the project, I receive a folder mapping error.

Perhaps this can be done with the, "Use current changeset for the triggered build" setting. I would like some way to specify a root folder from which to get changes. The unshelved changes for the build would only descend from the root folder specified.

Does this make it any clearer? Thanks!

[huserben]

Hi @jfinnegan3

Yes that makes it more clear to me. However what I don't know is: can the GetSources Task be "parametrized" with Variables and customized to only do partial unshelves? (I'm asking because I have faced similar problems with Gated Check-ins that fail because some other part of the code that is not under the mapped sources were part of the changeset).

The "use current Changeset" option just specifies a parameter ("sourceVersion") that is sent to VSTS/TFS as part of the request body. What happens with that parameter is then part of the VSTS implementation. It's basically the same as if you would manually queue a build in the web interface and specify the Label or Shelveset to build it with:

If however you could use/customize the GetSources task you could pass in a variable to the Build that is triggered that takes this and does it's logic based on that. Or you write a script that as part of your check-in shelves a subset of the "original" shelveset created as part of the gate and then triggers the "Sub-Build" with the newly created shelveset as source version. So if you get that far that you would have a specific shelveset with only the subset of the changes included the triggering of the other build could be configured in a way to only use this specific shelve.

Does this help you? I'm not sure how far I can assist directly with the Task, however I'm happy to help if you have more questions.

[jfinnegan3]

Yea, it does - that's certainly good insight. I don't know the exact variable that the unshelve command is run on, otherwise I'd alter it.

As I wrote, I tried get it to work by hacking your "Use same source branch" field (by entering: $(Build.SourceBranchName) ), but I believe it breaks because I added a space, so I tried encoding a space, but that didn't work either.

Is it possible you can disable any validation checks in that field and try to simulate a build by entering "$(Build.SourceBranchName) " which theoretically would run using a subset of a shelveset? Then check the triggered build and see what gets unshelved?

Thanks!

[huserben]

So I just tried the following: I used PostMan to manually send the Post Request that triggers the build by specifying the sourceBranch parameter as whatever I want:

It does some checking on the server and might return you an error when queuing - however with the request above it worked and triggered the Build.

In there it then failed during GetSources:

I googled a bit and came upon the documentation of the unshelve command: https://docs.microsoft.com/en-us/vsts/tfvc/unshelve-command?view=vsts

According to this it should be possible to specifiy the itemSpec that

Identifies the file or folder revisions to unshelve into the current workspace

The problem now is, the SourceVersion seems to set exactly the shelvesetname parameter (as you see what I passed via the parameter is enclosed by double-quotes). I quickly made a search if the source-code of the GetSources step are available to see whether it could be configured, but I did not manage to find anything.

As well I don't have the tf commandline tool at hand to try to test it locally.

So we could try to figure out how GetSources works and see if we can somehow configure it. Otherwise we might open an issue on a Microsoft Repo regarding this.

I will check around a bit online if I find something on this, but I cannot make any promises.

[jfinnegan3]

I also did some searching online, but I could not find any way to do it, and was thinking that your task might be setting a variable behind the scenes that I was unable to manipulate.

Perhaps it is impossible to do this using the API

[huserben]

I'll be in vacation the next few days, but when I find the time somewhen next week I will try to see whether I can open an Issue in a repo of Microsoft and ask whether it's possible or not.

I will update you in case anything comes back.

[huserben]

I did some additional search, but was without success. Now I'm trying my luck on StackOverflow (https://stackoverflow.com/questions/50395647/getting-only-partial-shelveset-in-vnext-build), I hope that will bring at least clarity whether its possible or not.

[huserben]

So just a quick update. It seems that it's not possible. I got linked to the following issue https://github.com/Microsoft/vsts-agent/issues/1087 and a uservoice that asks for the same https://visualstudio.uservoice.com/forums/330519-team-services/suggestions/10034220-build-vnext-ignore-no-appropriate-mapping-for-sh.

I will therefore close the issue. In case I would see that the functionality is implemented, I could always reopen the issue.