search

SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.24k stars 754 forks source link

Tests executed but empty specs analysis report #1335

Closed rdagumampan closed 5 years ago

rdagumampan commented 5 years ago

SpecFlow Version:

Used Test Runner

Version number:

Visual Studio Version

Are the latest Visual Studio updates installed?

.NET Framework:

Test Execution Method:

<SpecFlow> Section in app.config

<?xml version="1.0" encoding="utf-8"?>
<!--For more information on using transformations see the web.config examples at http://go.microsoft.com/fwlink/?LinkId=214134. -->
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <specFlow xdt:Transform="Replace">
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
    <!-- use unit test provider SpecRun+NUnit or SpecRun+MsTest for being able to execute the tests with SpecRun and another provider -->
    <unitTestProvider name="SpecRun+MsTest" />
    <!-- <unitTestProvider name="MsTest" /> -->
    <!-- <unitTestProvider name="MsTest" /> -->
    <!-- <unitTestProvider name="nunit" /> -->
    <!-- <unitTestProvider name="SpecRun+NUnit" /> -->
    <plugins>
      <add name="SpecRun" />
    </plugins>
  </specFlow>
</configuration>

Issue Description

Steps to Reproduce

2018-11-11T04:26:05.3741204Z ##[section]Starting: Run regression tests 2018-11-11T04:26:05.3741204Z ============================================================================== 2018-11-11T04:26:05.3741204Z Task : Visual Studio Test 2018-11-11T04:26:05.3741204Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2). 2018-11-11T04:26:05.3741204Z Version : 2.142.12 2018-11-11T04:26:05.3741204Z Author : Microsoft Corporation 2018-11-11T04:26:05.3741204Z Help : More information 2018-11-11T04:26:05.3741204Z ============================================================================== 2018-11-11T04:26:05.9981244Z SystemVssConnection exists true 2018-11-11T04:26:06.1385253Z SystemVssConnection exists true 2018-11-11T04:26:06.2321259Z Running tests using vstest.console.exe runner. 2018-11-11T04:26:06.2321259Z ====================================================== 2018-11-11T04:26:06.2321259Z Test selector : Test assemblies 2018-11-11T04:26:06.2321259Z Test filter criteria : TestCategory!=WIP 2018-11-11T04:26:06.2321259Z Search folder : D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev 2018-11-11T04:26:06.2477260Z VisualStudio version selected for test execution : latest 2018-11-11T04:26:06.2789262Z Run in parallel : false 2018-11-11T04:26:06.2789262Z Run in isolation : false 2018-11-11T04:26:06.2789262Z Path to custom adapters : D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev 2018-11-11T04:26:06.2789262Z Other console options : null 2018-11-11T04:26:06.2789262Z Code coverage enabled : false 2018-11-11T04:26:06.2789262Z Diagnostics enabled : false 2018-11-11T04:26:06.2789262Z SystemVssConnection exists true 2018-11-11T04:26:06.2789262Z Run the tests locally using vstest.console.exe 2018-11-11T04:26:06.2789262Z ======================================================== 2018-11-11T04:26:06.2789262Z Test selector : Test assemblies 2018-11-11T04:26:06.2789262Z Test assemblies : *RegressionTests*.dll,!\obj** 2018-11-11T04:26:06.2789262Z Test filter criteria : TestCategory!=WIP 2018-11-11T04:26:06.2789262Z Search folder : D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev 2018-11-11T04:26:06.2789262Z Run settings file : D:\AgentSPW1_work\r80\a 2018-11-11T04:26:06.2789262Z Run in parallel : false 2018-11-11T04:26:06.2789262Z Run in isolation : false 2018-11-11T04:26:06.2945263Z Path to custom adapters : D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev 2018-11-11T04:26:06.2945263Z Other console options : null 2018-11-11T04:26:06.2945263Z Code coverage enabled : false 2018-11-11T04:26:06.2945263Z Diagnostics enabled : false 2018-11-11T04:26:06.2945263Z Rerun failed tests: false 2018-11-11T04:26:06.2945263Z VisualStudio version selected for test execution : latest 2018-11-11T04:26:06.3881269Z ======================================================== 2018-11-11T04:26:06.4037270Z ====================================================== 2018-11-11T04:26:06.4193271Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" @D:\AgentSPW1_work_temp\e822c641-e569-11e8-814e-ef989b01f029.txt 2018-11-11T04:26:06.6377285Z Microsoft (R) Test Execution Command Line Tool Version 15.7.2 2018-11-11T04:26:06.6377285Z Copyright (c) Microsoft Corporation. All rights reserved. 2018-11-11T04:26:06.6377285Z 2018-11-11T04:26:06.6533286Z vstest.console.exe 2018-11-11T04:26:06.6533286Z "D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev\O.WP.Service.RegressionTests.dll" 2018-11-11T04:26:06.6533286Z /TestCaseFilter:"TestCategory!=WIP" 2018-11-11T04:26:06.6533286Z /logger:"trx" 2018-11-11T04:26:06.6533286Z /TestAdapterPath:"D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev" 2018-11-11T04:26:06.8405298Z Starting test execution, please wait... 2018-11-11T04:26:07.6829352Z SpecFlow+Runner execution started 2018-11-11T04:26:07.6829352Z SpecFlow+Runner 1.8.2 execution started 2018-11-11T04:26:07.6829352Z TestAdapter Location: D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev\packages\SpecRun.Runner.1.8.2\tools\TechTalk.SpecRun.VisualStudio.TestAdapter.dll 2018-11-11T04:26:07.6985353Z SpecRun: running tests in D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev\O.WP.Service.RegressionTests.dll 2018-11-11T04:26:07.6985353Z Profile: 2018-11-11T04:26:07.7297355Z Discovering target: Default 2018-11-11T04:26:08.5877410Z Passed LoadBalancer 2018-11-11T04:26:08.5877410Z Passed AddTwoNumbers 2018-11-11T04:26:09.1805448Z Result: tests executed with warnings 2018-11-11T04:26:09.1805448Z Total: 0 2018-11-11T04:26:09.1805448Z Succeeded: 0 2018-11-11T04:26:09.1805448Z Ignored: 0 2018-11-11T04:26:09.1805448Z Pending: 0 2018-11-11T04:26:09.1805448Z Skipped: 0 2018-11-11T04:26:09.1805448Z Failed: 0 2018-11-11T04:26:09.1805448Z 2018-11-11T04:26:09.1805448Z Execution Time: 00:00:00.0312002 2018-11-11T04:26:09.1805448Z Report file: file:///D:\AgentSPW1_work\r80\a\TestResults\Unnamed%20project2018-11-11T052607.html 2018-11-11T04:26:09.1961449Z Log file: file:///D:\AgentSPW1_work\r80\a\TestResults\Unnamed%20project2018-11-11T052607.log 2018-11-11T04:26:09.1961449Z SpecFlow+Runner execution finished 2018-11-11T04:26:09.3365458Z Results File: D:\AgentSPW1_work\r80\a\TestResults\teamcityagent_CSI00709_2018-11-11_05_26_08.trx 2018-11-11T04:26:09.3365458Z 2018-11-11T04:26:09.3365458Z Attachments: 2018-11-11T04:26:09.3365458Z D:\AgentSPW1_work_temp\egt24hot.4jp\Unnamed project2018-11-11T052607.log 2018-11-11T04:26:09.3365458Z D:\AgentSPW1_work_temp\egt24hot.4jp\Unnamed project2018-11-11T052607.html 2018-11-11T04:26:09.3365458Z 2018-11-11T04:26:09.3365458Z Total tests: 2. Passed: 2. Failed: 0. Skipped: 0. 2018-11-11T04:26:09.3365458Z Test Run Successful. 2018-11-11T04:26:09.3365458Z Test execution time: 2,3031 Seconds 2018-11-11T04:26:10.8809557Z ##[section]Async Command Start: Publish test results 2018-11-11T04:26:11.4737595Z Publishing test results to test run '56663' 2018-11-11T04:26:11.4737595Z Test results remaining: 2. Test run id: 56663 2018-11-11T04:26:11.8793621Z Published Test Run : https://tfsprodweusu4.visualstudio.com/A287824eb-a1bc-45dd-b17a-cb2942704169/DSS/_TestManagement/Runs#runId=56663&_a=runCharts 2018-11-11T04:26:11.8793621Z ##[section]Async Command End: Publish test results 2018-11-11T04:26:11.8949622Z ##[section]Finishing: Run regression tests

rdagumampan commented 5 years ago

image

rdagumampan commented 5 years ago

image

rdagumampan commented 5 years ago

Log file:

2018-11-11T05:26:07.6673351+01:00;VSTestExecutionThread;Info;SpecFlow+Runner execution started 2018-11-11T05:26:07.6829352+01:00;VSTestExecutionThread;Info;SpecFlow+Runner 1.8.2 execution started 2018-11-11T05:26:07.6829352+01:00;VSTestExecutionThread;Info;TestAdapter Location: D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev\packages\SpecRun.Runner.1.8.2\tools\TechTalk.SpecRun.VisualStudio.TestAdapter.dll 2018-11-11T05:26:07.6985353+01:00;VSTestExecutionThread;Info;SpecRun: running tests in D:\AgentSPW1_work\r80\a_BasicDotnetCoreServiceTemplate - RT\dev\O.WP.Service.RegressionTests.dll 2018-11-11T05:26:07.6985353+01:00;VSTestExecutionThread;Info;Profile: 2018-11-11T05:26:07.7297355+01:00;Thread#11;Info;Discovering target: Default Test Suite Dump

0: Unnamed project/ -

End: Test Suite Dump Thread#0:Scheduler: get next test for thread 0 Thread#:Scheduler: at the end of normal test queue Thread#0:Scheduler: enumerator for thread is at end Thread#0:Scheduler: No more tests for thread Thread#-1:Scheduler: get next test for thread -1 Thread#:Scheduler: at the end of normal test queue Thread#-1:Scheduler: enumerator for thread is at end Total: 0 Succeeded: 0 Ignored: 0 Pending: 0 Skipped: 0 Failed: 0

SabotageAndi commented 5 years ago

You have configured SpecRun+MSTest as unit test provider. So SpecFlow is generating code- behind- files for SpecFlow+Runner and MSTest.

I think in your case, both classes for the 2 runners are executed. So you get for SpecRun 0 tests and 2 from MSTest. It would be interesting, why the SpecFlow+Runner is not finding your scenarios.

If you check the code-behind files, are there classes with attributes from TechTalk.SpecRun?

SabotageAndi commented 5 years ago

@rdagumampan do you have still this issue?

SabotageAndi commented 5 years ago

Closed because of inactivity. Please reopen if the issue rise again.

lock[bot] commented 5 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.