Open hankweber opened 1 month ago
ADO task is automating vstest.console to provide additional functionality. If you need a way to run all test cases in one sessions then that issues should be raised in https://github.com/microsoft/azure-pipelines-tasks.
ok, I think we are looking to have the same behavior outside the ado task. so we want to execute our tests with each test case in its own session. I'll poke around the pipeline task code and see if I can see where they're doing that so we can copy it
Description
We are observing differences in test exectuion behavior when executing with the vstest ado task versus just invoking vstest.console.exe manually. What we are seeing is that each batch of test cases is run in it's own test session when running via the ado task whereas all test cases are run in the same test session when running manually with vstest.console.exe.
Steps to reproduce
Run a pipeline (example here) that uses the vstest task and test cases will be run in their own test session. Run the equivalent runsettings/configuration manually with vstest.console.exe and all test cases will be run in the same test session.
Expected behavior
The test case/session behavior should be the same regardless invocation.
Actual behavior
The test case/session behavior is different between the vstest task and the vstest.console.exe
Diagnostic logs
Slice from vstest.console.exe.log Slice1-from-vstest-ado-task.log Slice2-from-vstest-ado-task.log
Environment