search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.48k stars 2.61k forks source link

[BUG]: VSTest Failure: No tests were discovered while attempting to rerun failed tests #18084

Closed ismael-ms closed 3 months ago

ismael-ms commented 1 year ago

Task name

VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9

Task version

2.219.0

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Microsoft Windows NT 6.2.9200.0

Task log

2023-04-04T22:54:01.1805798Z ##[section]Starting: Run Framework Integration Tests
2023-04-04T22:54:01.2060851Z ==============================================================================
2023-04-04T22:54:01.2061199Z Task         : Visual Studio Test
2023-04-04T22:54:01.2061292Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2023-04-04T22:54:01.2061764Z Version      : 2.219.0
2023-04-04T22:54:01.2061812Z Author       : Microsoft Corporation
2023-04-04T22:54:01.2061866Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/vstest
2023-04-04T22:54:01.2061988Z ==============================================================================
2023-04-04T22:54:04.2951845Z Running tests using vstest.console.exe runner.
2023-04-04T22:54:04.2952218Z ======================================================

...

2023-04-04T22:55:51.5353894Z **************** Rerunning failed tests for Test run 1420630129 *********************
2023-04-04T22:55:51.5354457Z Max attempts: 3; Current attempt: 1; Failed test cases threshold: 20; Failed test cases max limit: 0
2023-04-04T22:55:51.5355351Z [command]C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "@D:\a\_temp\fz5rargpawg.tmp"
2023-04-04T22:55:51.6336102Z Microsoft (R) Test Execution Command Line Tool Version 16.11.0
2023-04-04T22:55:51.6336780Z Copyright (c) Microsoft Corporation.  All rights reserved.
2023-04-04T22:55:51.6406081Z vstest.console.exe /TestCaseFilter:"FullyQualifiedName=Microsoft.Azure.Maps.DataPipeline.StatusManagementIntegrationTests.StatusManagerTests.StatusManagement_InitialJobTimesOutButStillCompletes (Succeeded)"
2023-04-04T22:55:51.6406794Z "D:{redacted}.dll"
2023-04-04T22:55:51.6407904Z /Settings:"D:\a\_temp\4k2fcp52xtu.tmp.runsettings"
2023-04-04T22:55:51.6408353Z /Logger:"trx"
2023-04-04T22:55:51.6408738Z /TestAdapterPath:"D:{redacted}\Tests"
2023-04-04T22:55:52.2944742Z Starting test execution, please wait...
2023-04-04T22:55:53.8144268Z 2.707
2023-04-04T22:55:53.9523289Z A total of 1 test files matched the specified pattern.
2023-04-04T22:55:59.4927275Z ##[error]Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
2023-04-04T22:55:59.7278102Z No test matches the given testcase filter `FullyQualifiedName=Microsoft.Azure.Maps.DataPipeline.StatusManagementIntegrationTests.StatusManagerTests.StatusManagement_InitialJobTimesOutButStillCompletes (Succeeded)` in D:{redacted}.dll
2023-04-04T22:56:00.1607663Z Results File: D:\a\_temp\TestResults\VssAdministrator_fv-az364-167_2023-04-04_22_56_00.trx
2023-04-04T22:56:00.1708287Z ##[error]Test Run Failed.
2023-04-04T22:56:00.1907680Z Vstest.console.exe exited with code 1.
2023-04-04T22:56:00.1908252Z **************** Completed test execution *********************
2023-04-04T22:56:00.1956047Z Test results files: D:\a\_temp\TestResults\VssAdministrator_fv-az364-167_2023-04-04_22_56_00.trx
2023-04-04T22:56:00.2045861Z No Result Found to Publish 'D:\a\_temp\TestResults\VssAdministrator_fv-az364-167_2023-04-04_22_56_00.trx'.
2023-04-04T22:56:00.2046510Z Updating test results: 0
2023-04-04T22:56:00.2464521Z Publishing test results to test run '1420630129'.
2023-04-04T22:56:00.4364334Z TestResults To Publish 0, Test run id:1420630129
2023-04-04T22:56:00.4365149Z Updated test results: 0
2023-04-04T22:56:00.4370959Z Publishing Attachments: 1
2023-04-04T22:56:00.6023453Z ##[error]Error occurred while publishing test results : System.Exception: No tests were discovered while attempting to rerun failed tests. This is likely to happen if these tests are xunit or nunit datadriven/parameterized tests or tests with custom display names with spaces or special characters which are not supported with rerun. Please reach out to us at https://github.com/microsoft/azure-pipelines-tasks/issues if you are hitting this for tests other than the ones mentioned before.
2023-04-04T22:56:00.7415454Z Execution Result Code 1 is non zero, checking for failed results
2023-04-04T22:56:07.2025868Z Retry count is exhausted while performing System.Threading.Tasks.Task`1[Microsoft.TeamFoundation.TestManagement.WebApi.TestRunStatistic] <GetTestRunSummaryByOutcome>b__0()
2023-04-04T22:56:07.2196457Z Failure attempting to call the test run summary and retry counter is exhausted Microsoft.TeamFoundation.DistributedTask.Task.TestResults.RetryCountExhaustedException: Max retries count: 3 is exhausted ---> Microsoft.TeamFoundation.TestManagement.WebApi.TestObjectNotFoundException: Run summary is not available, It is recommended to try again after 1 sec.
2023-04-04T22:56:07.2197308Z    at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<HandleResponseAsync>d__53.MoveNext()
2023-04-04T22:56:07.2197656Z --- End of stack trace from previous location where exception was thrown ---
2023-04-04T22:56:07.2198167Z    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2023-04-04T22:56:07.2198508Z    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2023-04-04T22:56:07.2198882Z    at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__51.MoveNext()
2023-04-04T22:56:07.2199359Z --- End of stack trace from previous location where exception was thrown ---
2023-04-04T22:56:07.2199678Z    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2023-04-04T22:56:07.2199985Z    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2023-04-04T22:56:07.2200790Z    at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__47`1.MoveNext()
2023-04-04T22:56:07.2201114Z --- End of stack trace from previous location where exception was thrown ---
2023-04-04T22:56:07.2201393Z    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2023-04-04T22:56:07.2201720Z    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2023-04-04T22:56:07.2202933Z    at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__28`1.MoveNext()
2023-04-04T22:56:07.2203263Z --- End of stack trace from previous location where exception was thrown ---
2023-04-04T22:56:07.2203544Z    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2023-04-04T22:56:07.2203868Z    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2023-04-04T22:56:07.2204199Z    at Microsoft.TeamFoundation.TestClient.PublishTestResults.TestRunPublisher.<GetTestRunSummaryAsync>d__9.MoveNext()
2023-04-04T22:56:07.2204724Z --- End of stack trace from previous location where exception was thrown ---
2023-04-04T22:56:07.2205196Z    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2023-04-04T22:56:07.2205544Z    at Microsoft.TeamFoundation.DistributedTask.Task.TestResults.RetryHelper.<Retry>d__1`1.MoveNext()
2023-04-04T22:56:07.2205900Z    --- End of inner exception stack trace ---
2023-04-04T22:56:07.2206251Z    at Microsoft.TeamFoundation.DistributedTask.Task.TestResults.RetryHelper.<Retry>d__1`1.MoveNext()
2023-04-04T22:56:07.2206942Z --- End of stack trace from previous location where exception was thrown ---
2023-04-04T22:56:07.2207235Z    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2023-04-04T22:56:07.2207580Z    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2023-04-04T22:56:07.2207947Z    at Microsoft.TeamFoundation.DistributedTask.Task.TestResults.TestRunSummaryManager.<GetTestRunSummaryByOutcome>d__3.MoveNext()
2023-04-04T22:56:07.2325552Z ##[warning]Unable to fetch test run summary from the Test Management Server due to exception: One or more errors occurred.
2023-04-04T22:56:07.2330147Z PERF WARNING: TestRunCompletionPhase:GetTestRunOutcomeFromPublishTestResultLibrary: took 6398.4196 ms
2023-04-04T22:56:07.3239053Z Completed TestExecution Model...
2023-04-04T22:56:07.3496525Z ##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
2023-04-04T22:56:07.3498140Z ##[error]Error: The process 'D:\a\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.219.0\Modules\DTAExecutionHost.exe' failed with exit code 1
2023-04-04T22:56:07.3508637Z ##[error]Vstest failed with error. Check logs for failures. There might be failed tests.
2023-04-04T22:56:07.3624343Z ##[section]Finishing: Run Framework Integration Tests

Relevant log output

2023-04-04T22:55:51.5353894Z **************** Rerunning failed tests for Test run 1420630129 *********************
2023-04-04T22:55:51.5354457Z Max attempts: 3; Current attempt: 1; Failed test cases threshold: 20; Failed test cases max limit: 0
2023-04-04T22:55:51.5355351Z [command]C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "@D:\a\_temp\fz5rargpawg.tmp"
2023-04-04T22:55:51.6336102Z Microsoft (R) Test Execution Command Line Tool Version 16.11.0
2023-04-04T22:55:51.6336780Z Copyright (c) Microsoft Corporation.  All rights reserved.
2023-04-04T22:55:51.6406081Z vstest.console.exe /TestCaseFilter:"FullyQualifiedName=Microsoft.Azure.Maps.DataPipeline.StatusManagementIntegrationTests.StatusManagerTests.StatusManagement_InitialJobTimesOutButStillCompletes (Succeeded)"
2023-04-04T22:55:51.6406794Z "D:{redacted}.dll"
2023-04-04T22:55:51.6407904Z /Settings:"D:\a\_temp\4k2fcp52xtu.tmp.runsettings"
2023-04-04T22:55:51.6408353Z /Logger:"trx"
2023-04-04T22:55:51.6408738Z /TestAdapterPath:"D:{redacted}\Tests"
2023-04-04T22:55:52.2944742Z Starting test execution, please wait...
2023-04-04T22:55:53.8144268Z 2.707
2023-04-04T22:55:53.9523289Z A total of 1 test files matched the specified pattern.
2023-04-04T22:55:59.4927275Z ##[error]Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
2023-04-04T22:55:59.7278102Z No test matches the given testcase filter `FullyQualifiedName=Microsoft.Azure.Maps.DataPipeline.StatusManagementIntegrationTests.StatusManagerTests.StatusManagement_InitialJobTimesOutButStillCompletes (Succeeded)` in D:{redacted}.dll
2023-04-04T22:56:00.1607663Z Results File: D:\a\_temp\TestResults\VssAdministrator_fv-az364-167_2023-04-04_22_56_00.trx
2023-04-04T22:56:00.1708287Z ##[error]Test Run Failed.
2023-04-04T22:56:00.1907680Z Vstest.console.exe exited with code 1.
2023-04-04T22:56:00.1908252Z **************** Completed test execution *********************
2023-04-04T22:56:00.1956047Z Test results files: D:\a\_temp\TestResults\VssAdministrator_fv-az364-167_2023-04-04_22_56_00.trx
2023-04-04T22:56:00.2045861Z No Result Found to Publish 'D:\a\_temp\TestResults\VssAdministrator_fv-az364-167_2023-04-04_22_56_00.trx'.
2023-04-04T22:56:00.2046510Z Updating test results: 0
2023-04-04T22:56:00.2464521Z Publishing test results to test run '1420630129'.
2023-04-04T22:56:00.4364334Z TestResults To Publish 0, Test run id:1420630129
2023-04-04T22:56:00.4365149Z Updated test results: 0
2023-04-04T22:56:00.4370959Z Publishing Attachments: 1
2023-04-04T22:56:00.6023453Z ##[error]Error occurred while publishing test results : System.Exception: No tests were discovered while attempting to rerun failed tests. This is likely to happen if these tests are xunit or nunit datadriven/parameterized tests or tests with custom display names with spaces or special characters which are not supported with rerun. Please reach out to us at https://github.com/microsoft/azure-pipelines-tasks/issues if you are hitting this for tests other than the ones mentioned before.

Aditional info

We have experienced our test pipeline fail intermittently due to this issue.
MBGoldberg-BD commented 1 year ago

Looks like the same issue https://github.com/microsoft/azure-pipelines-tasks/issues/16990 we're still waiting to be resolved.

joshua4163 commented 1 year ago

Pls use *\$(BuildConfiguration)*\tests.dll under VsTest-testAssemblies -->paste under Test files.It will start taking all the tests.i am talking about classic pipeline.

github-actions[bot] commented 11 months ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

MBGoldberg-BD commented 11 months ago

Still fails for us on Classic Pipeline . VsTest3. Looks like the same issue https://github.com/microsoft/azure-pipelines-tasks/issues/16990 we're still waiting to be resolved.

github-actions[bot] commented 5 months ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days