Move "Post-Job PostBuildCleanup" task after "Post-Job Checkout" task

[HackproTm]

I have a pipeline with multiple checkout repositories, I add the PostBuildCleanUp task to clean the Source Directories, but my pipeline is failing due to a compatibility issue between the "checkout" task and the "PostBuildCleanup" task

• Extension: PostBuildCleanup
• Environment: Azure DevOps Services (cloud)
• Agent type: self-hosted
  • Agent version: 2.195.2
• Pipeline type: yaml pipeline

The task PostBuildCleanup create automatically a Post-Job task to Clean some directories into the Pool Agent, the problem here is due a this Post-Job is not the last task, the task "checkout" also adds another Post-Job task and this task tries to run some command into the directories that Post-Job PostBuildCleanup task delete previously. This issue breaks the pipeline.

Please move the Post Build Cleanup task to last

Task logs Run your pipeline with the following variables:

• For PostBuildCleanup: System.Debug and PBC.LogRawData set to true

Post-job: PostBuildCleanup LOGS

##[debug]Evaluating condition for step: 'Clean Agent Directories'
##[debug]Evaluating: AlwaysNode()
##[debug]Evaluating AlwaysNode:
##[debug]=> True
##[debug]Result: True
Starting: Clean Agent Directories
==============================================================================
Task         : Post Build Cleanup
Description  : Cleans the build directories on the agent after the build has finished.
Version      : 3.1.1
Author       : Microsoft Premier Services
Help         : [More information](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/PostBuildCleanup/en-US/overview.md)
==============================================================================
##[debug]Using node path: E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\node\bin\node.exe
##[debug]agent.TempDirectory=E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\_temp
##[debug]loading inputs and endpoints
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading INPUT_DISABLECERTCHECK
##[debug]loading SECRET_SONARQUBE_ENDPOINT
##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
##[debug]loading VSTS_TASKVARIABLE_CLEANUP
##[debug]loaded 7
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]check path : E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\_tasks\PostBuildCleanup_XXXXXXXXXXXXXXXXXX\3.1.1\task.json
##[debug]adding resource file: E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\_tasks\PostBuildCleanup_XXXXXXXXXXXXXXXXXX\3.1.1\task.json
##[debug]system.culture=en-US
##[debug]Agent.Version=2.195.2
##[debug]task variable: cleanup=true
##[debug]disableCertCheck=false
##[debug]SystemVssConnection exists true
##[debug]System.TeamFoundationCollectionUri=https://dev.azure.com/XXXXXXXXXXXXXXXXXX/
##[debug]PBC.LogRawData=true
##[debug]System.TeamProject=XXXXXXXXXXXXXXXXXX
##[debug]System.DefinitionId=XXXXXXXXXXXXXXXXXX
##[debug]Build.DefinitionVersion=8
##[debug]Build.BuildId=XXXXXXXXXXXXXXXXXX
##[debug]System.JobId=XXXXXXXXXXXXXXXXXX
##[debug]System.PhaseDisplayName=XXXXXXXXXXXXXXXXXX
##[debug]System.JobDisplayName=XXXXXXXXXXXXXXXXXX
##[debug]PSGer.Build.PollInterval=undefined
##[debug]PSGer.Build.MaxWaitTime=undefined
##[debug]getCurrentRepositoryCleanOption
##[debug]prepareClients
##[debug]Build.Repository.Clean=True
##[debug]waitForPreviousBuildStepsToFinish
##[debug]getPreviousBuildSteps
##[debug]prepareClients
Cleaning up $(Agent.BuildDirectory)...
##[debug]Build.BinariesDirectory=E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\20\b
##[debug]Build.SourcesDirectory=E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\20\s
##[debug]Build.StagingDirectory=E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\20\a
##[debug]Common.TestResultsDirectory=E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\20\TestResults
##[debug]Agent.BuildDirectory=E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\20
##[debug]deleteAllContent
##[debug]Agent.BuildDirectory=E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\20
##[debug]deleteAllContent
Finishing: Clean Agent Directories

Post-job: Checkout Repository

##[debug]Evaluating condition for step: 'Checkout Repository XXXXXXXXXXXXXXXXXX'
##[debug]Evaluating: AlwaysNode()
##[debug]Evaluating AlwaysNode:
##[debug]=> True
##[debug]Result: True
Starting: Checkout Repository XXXXXXXXXXXXXXXXXX
==============================================================================
Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version      : 1.0.0
Author       : Microsoft
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
==============================================================================
Cleaning any cached credential from repository: XXXXXXXXXXXXXXXXXX (git)
##[debug]Repository url=https://XXXXXXXXXXXXXXXXXX@dev.azure.com/XXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXX/_git/XXXXXXXXXXXXXXXXXX
##[debug]targetPath=E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\20\s\XXXXXXXXXXXXXXXXXX
Prepending Path environment variable with directory containing 'git.exe'.
##[debug]Processed: ##vso[task.prependpath]E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\git\mingw64\bin
##[debug]Processed: ##vso[task.prependpath]E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\git\cmd
##[debug]PATH: 'E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\git\cmd;E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\git\mingw64\bin;C:\Windows\ServiceProfiles\NetworkService\.dotnet\tools;E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\_tool/dotnet;E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\git\cmd;E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\git\mingw64\bin;C:\Python27\;C:\Python27\Scripts;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Puppet Labs\Puppet\bin;C:\ProgramData\chocolatey\bin;C:\Program Files\dotnet\;C:\Program Files (x86)\dotnet\;C:\Program Files\Java\jdk1.8.0_211\bin;C:\Program Files\Java\jdk1.8.0_211\bin;C:\Program Files\Git\cmd;C:\Program Files\nodejs\;C:\Program Files\Microsoft SQL Server\150\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn\;C:\Program Files\Azure Data Studio\bin;C:\Program Files\Java\jdk1.8.0_211\bin;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;C:\Program Files\Microsoft VS Code\bin;C:\Program Files\PowerShell\7\;C:\Windows\ServiceProfiles\NetworkService\AppData\Local\Microsoft\WindowsApps'
##[debug]Get git version.
git version
##[debug]Starting process:
##[debug]  File name: 'E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\git\cmd\git.exe'
##[debug]  Arguments: 'version'
##[debug]  Working directory: 'E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work'
##[debug]  Require exit code zero: 'False'
##[debug]  Encoding web name: utf-8 ; code page: '65001'
##[debug]  Force kill process on cancellation: 'False'
##[debug]  Redirected STDIN: 'False'
##[debug]  Persist current code page: 'False'
##[debug]  Keep redirected STDIN open: 'False'
##[debug]  High priority process: 'False'
##[debug]OOM score adjustment is Linux-only.
##[debug]Process started with process id 27256, waiting for process exit.
##[debug]STDOUT/STDERR stream read finished.
##[debug]Exited process 20388 with exit code 0
##[debug]Finished process 20388 with exit code 0, and elapsed time 00:00:00.4859900.
git-lfs/2.13.3 (GitHub; windows amd64; go 1.16.2; git a5e65851)
##[debug]Detect git-lfs version: '2.13.3'.
##[debug]Set git useragent to: git/2.30.2 (vsts-agent-git/2.195.2).
##[debug]Remove any extraheader, proxy and client cert setting from git config.
##[debug]Unset git config --unset-all http.https://XXXXXXXXXXXXXXXXXX@dev.azure.com/XXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXX/_git/XXXXXXXXXXXXXXXXXX.extraheader
git config --unset-all http.https://XXXXXXXXXXXXXXXXXX@dev.azure.com/XXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXX/_git/XXXXXXXXXXXXXXXXXX.extraheader
##[debug]Starting process:
##[debug]  File name: 'E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\externals\git\cmd\git.exe'
##[debug]  Arguments: 'config --unset-all http.https://XXXXXXXXXXXXXXXXXX@dev.azure.com/XXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXX/_git/XXXXXXXXXXXXXXXXXX.extraheader'
##[debug]  Working directory: 'E:\ADOAgents\XXXXXXXXXXXXXXXXXX-1\_work\20\s\XXXXXXXXXXXXXXXXXX'
##[debug]  Require exit code zero: 'False'
##[debug]  Encoding web name: utf-8 ; code page: '65001'
##[debug]  Force kill process on cancellation: 'False'
##[debug]  Redirected STDIN: 'False'
##[debug]  Persist current code page: 'False'
##[debug]  Keep redirected STDIN open: 'False'
##[debug]  High priority process: 'False'
##[error]The directory name is invalid.
##[debug]Processed: ##vso[task.logissue type=error;]The directory name is invalid.
##[debug]Processed: ##vso[task.complete result=Failed;]
##[debug]   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
   at Agent.Plugins.Repository.GitCliManager.ExecuteGitCommandAsync(AgentTaskPluginExecutionContext context, String repoRoot, String command, String options, CancellationToken cancellationToken)
   at Agent.Plugins.Repository.GitCliManager.GitConfigUnset(AgentTaskPluginExecutionContext context, String repositoryPath, String configKey)
   at Agent.Plugins.Repository.GitSourceProvider.RemoveGitConfig(AgentTaskPluginExecutionContext executionContext, GitCliManager gitCommandManager, String targetPath, String configKey, String configValue)
   at Agent.Plugins.Repository.GitSourceProvider.PostJobCleanupAsync(AgentTaskPluginExecutionContext executionContext, RepositoryResource repository)
   at Agent.Plugins.Repository.CleanupTask.RunAsync(AgentTaskPluginExecutionContext executionContext, CancellationToken token)
   at Agent.PluginHost.Program.Main(String[] args)
Finishing: Checkout Repository XXXXXXXXXXXXXXXXXX

[ReneSchumacher]

Hi @HackproTm,

sorry for the late response, this has somehow slipped my attention. I can see that you're using the task in a YAML pipeline, which doesn't really work. Due to some major changes in how YAML pipelines store the clean information, the task isn't fully able to understand what to delete. In addition, the task doesn't know how to handle multiple repositories. In your screenshot I can see multiple post-job checkout steps which leads to the assumption that you have multiple checkout steps in your pipeline.

Otherwise, the clean option would work. We handle the .git folder in a very special way to ensure that the post-job checkout step also works when our cleanup task is enabled.

Unfortunately, there is no way to move the cleanup logic after the post-job checkout logic. A task can only require to run a post-job script but these scripts are always run in reverse order of the tasks. Since the checkout task is always the very first task, its post-job step automatically runs as the very last step.

Since it will be very hard to create a YAML-ready post build cleanup task, I'm currently thinking of publishing a couple YAML template that can achieve the same things. I'll let you know when these are ready.

[HackproTm]

Hi @ReneSchumacher

I found a workaround to solve this issue. I added the "Post Build Clean" task as the first task in our pipeline, also I forced the checkout "self" repository as an additional task (overriding the default behavior). Something like this:

This put the PostJob task as the last task. And now the checkout task doesn't fail.

Thanks for taking the time to respond.

Best regards