Test Agent Deployment Fails using v2 2017 Agents

[cytron201]

Switched an existing on-prem Automation UI build to use 2017.1 v2 agent deployment step. Was previously working on 2017 using v1 step.

It errors saying "##[error]Error occured on 'ORBUS-AGENT-04:5985'. Details : 'The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Cannot find path 'C:\TestAgent2017Setup\vstf_testagent.exe' because it does not exist."

The build step is set to download and upgrade the agent when it runs. I've logged in and uninstalled the 2015 agent from the target machine incase there was a conflict. I can confirm that the 2017 agent is in the "C:\TestAgent" folder as expected, which is being populated by this step.

Confused as to why it then complains about this totally different folder. Logs for this step are attached.

10_Deploy Test Agents.txt

[nigurr]

@cytron201 are you using TFS OnPrem or VSTS? If you are using TFS 2017 U1, This issue is fixed in Update 2 which will be released soon. Alternatively, you can patch the task in your TFS from latest master using tfx-cli [We can guide you through the steps].

PR for ref: https://github.com/Microsoft/vsts-tasks/pull/3707/files

Sorry for the trouble.

[cytron201]

@nigurr hi, yes it's 2017.1 on prem.

I've edited the InstallTestAgent.ps1 in the Build Agent work\tasks folder to remove the lines you deleted in your change.

I am now getting a new error:

Unable to start DTAExecutionHost process

[cytron201]

@nigurr The new issue seems to be the same as the one described here: https://github.com/Microsoft/vsts-tasks/issues/3837

The work around is to RDP the machine while the agent is being installed. We have a farm of 20 UI Automation VMs, and this is therefore not really feasible. Is there a better fix available yet please?

[cytron201]

@nigurr Reducing the target agents to one machine, and using the RDP workaround it is able to pass this step. It however fails on the Run Tests step, reporting that test files have not been copied across. Logs are attached: 11_Run Automated UI Tests.zip

I'm struggling to understand how this rolled out like this in a Production Build, is this not a common use of these steps or is VSTS the only environment tested comprehensively at this point? I understand the build step is in preview, but it's completely non functional at the moment.

[nigurr]

@cytron201 Its inexcusable. We will consider this feedback seriously and will make sure we ship with good quality. If you patch the task from latest master, even the second issue gets addressed automatically.

Are you using CodedUI 2017 for your tests? if not, you can continue to use 2015 Test agent

[cytron201]

@nigurr Yes we are trying to migrate to VS2017 CodedUI.

Thanks for the assistance - to run the latest version of the Tasks straight from Master, do i need to follow the instructions on this site: https://www.visualstudio.com/en-us/docs/integrate/extensions/develop/add-build-task?

How can i push these into TFS to replace the pre-installed versions?

[nigurr]

Please do make sure you take the backup of your existing tasks and database and try it in demo version/test version before applying in Production.

Here are the steps:

1. Install latest version of NodeJS and install https://github.com/Microsoft/tfs-cli
2. Clone the Tasks repository https://github.com/Microsoft/vsts-tasks - master branch
3. Build the tasks by executing node make.js build --task DeployVisualStudioTestAgent from C:\Vsts-tasks\
4. Once the build is complete, you will notice the following file structure C:\Vsts-tasks\_build\Tasks\DeployVisualStudioTestAgent\
5. Then patch the tasks by following executing tfx build tasks upload --task-path C:\Vsts-tasks\_build\Tasks\DeployVisualStudioTestAgent\ --overwrite --service-url http://YourTfsServer/tfs/DefaultCollection and it will ask you for Personal Access Token which you can generate from your TFS My Account -> Security page

[nigurr]

Closing this issue as it will be fixed and supported in TFS 2017 U2.

[pi3k14]

@nigurr How can you close a critical case that has not been solved? A month later there is still not released a fix for such a critical error!

[nigurr]

@pi3k14 For On-Prem TFS, the fix will be available via 2017 Update 2. For VSTS customers, the fix is already rolled out.

You can continue to use 2015 Test agent as we don't have any feature changes from 2015 TA to 2017 Test agent. Please let us know if you have any critical issue blocking. We will happy to resolve

[pi3k14]

@nigurr, 2015 Test agent has this problem https://github.com/Microsoft/vsts-tasks/issues/4101, or are your referring to the Deploy test agent v1.0 task?

[nigurr]

@pi3k14 can you please describe the issue you are facing?

1.using TA 2017 or TA 2015? 2.using DTA v1 or v2 task? 3.what's the exact issue you are facing?

[pi3k14]

@nigurr, I have a VS2017 testproject that I'm trying to run remotely. We have an On-Prem TFS, where I'm using the Deploy Test Agent v2 (preview) task. If I select the Visual Studio 2017 test agent for deployment it crash with this issue (#3922), if I select Visual Studio 2015 test agent for deployment it crash with issue #4101. The Deploy Test Agent v2 (preview) task seems dead in the water.

[nigurr]

@pi3k14, #4101 is related to Test Agent 2017. Can you please attach DTAExecutionHost log from C:\TestAgent\ folder?

You can continue to run VS 2017 tests using Test Agent 2015. You can just drop the mstest v2 adapter and the tests will run out of the box.

Alternatively, you can manually install Test agent 2017 for one time and continue to use the DTA v2 task with TA 2017.

[pi3k14]

@nigurr DTAExecutionHost.exe.txt version 15.111.26109.0 of DTAExecutionHost.exe

[nigurr]

@pi3k14 thanks for the logs. Is Test Agent 2017 and Test Agent 2015 both are located in the same machine?

[pi3k14]

@nigurr Yes, but quite frankly I don't understand the question. It is Test Agent Deploy that deploys the test agents, and yes I don't have different machines for running the tests on.

[nigurr]

@pi3k14 Sorry for the trouble. We don't support side by side installation of TA 2015 and TA 2017. [Temp Workaround till Update 2] If you have single test agent machine, You can manually install TA 2017 and use this machine for running the tests. The bug was there in installation of Test Agent. This can be bypassed using manual installation. If you want to use TA 2015 and run your tests, you need to uninstall TA 2017 and cleanup registry (I can help you with this) so that you can continue to run both VS 2017/2015 tests using the TA 2015.

[pi3k14]

@nigurr, trying to manually install TA 2017 from https://aka.ms/vs/15/release/vs_TestAgent.exe result in an install with zero progress.

A guess you have to give my instruction to cleanup registry for this.

[nigurr]

@pi3k14 is this machine connected to internet? If not we need to prepare the layout for the test agent from internet connected machine

[pi3k14]

@nigurr this machine is connected to the internet, but not for inbound traffic (it's a internal test machine). Are you implying that manual test agent installation is not possible?

[nigurr]

@pi3k14 it's possible. we published blog for the same https://blogs.msdn.microsoft.com/visualstudioalm/2017/05/05/using-visual-studio-agent-deployment-task-on-machines-not-connected-to-the-internet/

1. From the internet connected machine

vs_testagent.exe --layout C:\TA2017Setup --lang en-US

2. Copy the entire folder C:\TA2017Setup to Test Agent machine and run the setup

C:\TA2017Setup\vs_testagent.exe --installPath C:\TestAgent2017\

[pi3k14]

@nigurr sorry, but this give the same problem https://github.com/Microsoft/vsts-tasks/issues/3922#issuecomment-301699137 (no progress in installation)

[nigurr]

@pi3k14 can you drop your skype to devops_tools at microsoft dot com We will assist you with this.

[pi3k14]

@nigurr have to get back on this, end of day and tomorrow is a public holiday

[nigurr]

@pi3k14 I installed into one machine which is not connected to internet and it worked.

Followed the same step which I shared with you. Before installing, in the C:\Ta2017Setup\certificates -> there are couple of signed certificates. These must be installed [right click -> install] before starting installation.

[pi3k14]

@nigurr manually installing the certificates did the trick, thank you :)

But then I stumbled into the next problem, it doesn't seem like the Run Functional Tests Task support net core tests "Error : No tests were discovered from the specified test sources". I am using the NUnit3 TestAdapter v 3.8.0 (alpha1) http://www.alteridem.net/2017/05/04/test-net-core-nunit-vs2017/ , any pointers on how to proceed?

[nigurr]

@pi3k14 please copy the "Nunit adapaters" along with your Test Assemblies to remote machine. The engine will auto-load the adapters if it finds within test folder.

[pi3k14]

@nigurr yes I know. Both the NUnit3.TestAdapter.dll and the nunit.engine.netstandard.dll are copied. I even overrode the testrun parameter TestAdaptersPaths to point to the test folder, but it didn't change anything.

Is there any "hidden" parameters to get trace information from the Run Functional Tests Task?

[nigurr]

@pi3k14 There's no hidden information. We print everything to "C:\TestAgent\dtaexecutionhost.log". You can try running the tests with "vstest.console.exe" from test agent machine and see if tests are getting discovered. use "vstest.console.exe /?" for help. We use the same engine to execute the tests.

[pi3k14]

@nigurr found the problem in the vstest.console diagnostics

I, 4300, 1, 2017/05/18, 10:03:10.895, 5130358131693, vstest.console, AssemblyResolver: System.Runtime, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a: Resolving assembly.
V, 4300, 1, 2017/05/18, 10:03:10.895, 5130358133028, vstest.console, CurrentDomain_AssemblyResolve: Resolving assembly 'System.Runtime, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
V, 4300, 1, 2017/05/18, 10:03:10.895, 5130358133319, vstest.console, CurrentDomain_AssemblyResolve: Failed to resolve assembly 'System.Runtime, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

found several references to this problem at StackOverflow (eg. http://stackoverflow.com/a/41414568/2294065), but not sure how to proceed from here. Is this something that has to be fixed in vstest.console? Both NUnit (3.6.1) and Microsoft.NET.Test.Sdk (15.0) are bound to version 4.3 of System.Runtime

[prasadkumar1]

Hi Experts, I got struck with Release error in vsts + BizTalk Server 2016 with Feature pack installed. While creating release definition in VSTS I am getting below release error message, please help me to error message. when iam trying to multiple projects into the same solution for Automatically deploy BizTalk Server applications using Visual Studio Team Services and getting the below release error message.

"Cannot find path 'C:\agent_work\r3\a_btsDeploy\unzipped\SampleDemoApplication\bin\Binding.xml' because it does not exist.".