Closed sailesh89 closed 5 months ago
Related to https://github.com/microsoft/azure-pipelines-tasks/issues/19222. We have see it work and fail with with 2.229.0.
Working
Discovered tests 12348 from sources
=================================================================
[RunStatistics]This execution slice with id '168030', received '500' testcases to execute out of which '500' is discovered.
**************** Starting test execution *********************
Using new test platform for test execution
Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleWrapper instance will be created
Broken
Discovered tests 12371 from sources
=================================================================
[RunStatistics]This execution slice with id '168529', received '500' testcases to execute out of which '0' is discovered.
List of test cases in slice with id: 168529
Possible fix is to move from VSTest@2 to VSTest@3 (Undocumented https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vstest-v2?view=azure-pipelines)
Wonder if the node pathing has changed as it looks for the dlls? Could be due to https://github.com/microsoft/azure-pipelines-tasks/pull/19215
Possible fix is to move from VSTest@2 to VSTest@3 (Undocumented https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vstest-v2?view=azure-pipelines)
I have a similar issue where 2.229.0 was working (found and executed 6/6 test cases in a Test Plan), but now only runs 3/6 Test Cases and leaves the remaining test cases with an Outcome of 'In Progress'.
I have just tried VSTest@3 (v3.225.0), and now all 6 test cases are executed successfully again.
Possible fix is to move from VSTest@2 to VSTest@3 (Undocumented https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vstest-v2?view=azure-pipelines)
Wonder if the node pathing has changed as it looks for the dlls? Could be due to #19215
Upgrade won't work in our case as it will fail the Automated test run from the Test plan validation. Looks like its when we select a subset of tests to be run from test plan, choosing this release with vstest@v2 only works. Rest fails the validation and wont kick it off. Link to referenced steps from microsoft.
I did test with vstest@V3 to run entire test plan. It seems to be working with the v3 task. But unfortunately, the Automated test run validation is not allowing V3 task. Probably will need to fix that validation and we can try with the V3 task in the test run mode from Automated test run.
@qianz2 @merlynomsft @rasunkar any thoughts?
I have exactly the same setup as @sailesh89 and am experiencing the same issue.
On the afternoon of 9th November a number of tests stopped working. Now this morning (10th) those same tests are executing. No changes were made on my end at any point.
This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days
This still seems to be an issue with Version 2 of the vstest runner,
still unable to run tests from test plan with version 3
can we reopen this issue?
or have some form of update on this issue?
I stopped using VSTest it as it doesn't work on Ubuntu anyway. We use dotnet test now and our own custom version of https://github.com/idubnori/ParallelTestingSample-dotnet-core
Task name
VsTest
Task version
2.x
Environment type (Please select at least one enviroment where you face this issue)
Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Windows
Task log
Relevant log output
Aditional info