search

nspec / NSpec.VsAdapter

A test adapter to run NSpec tests from Test Explorer in Visual Studio 2013 and later.
http://nspec.org/
MIT License
4 stars 2 forks source link

In VS2015 after test discovery runs successfully, no test is listed in Test Explorer window #4

Closed BrainCrumbz closed 8 years ago

BrainCrumbz commented 8 years ago

Tried with SampleSpec solution. This is the output window log with trace min log level:

------ Discover test started ------
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery started
NSpec VS Adapter 0.1.0.0: [INFO]  Discovering tests in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\AdHocConsoleRunner\bin\Debug\AdHocConsoleRunner.exe'
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery started
NSpec VS Adapter 0.1.0.0: [INFO]  Discovering tests in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\ConfigSampleSpecs\bin\Debug\ConfigSampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [DEBUG] Start collecting tests in 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\ConfigSampleSpecs\bin\Debug\ConfigSampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [DEBUG] Start collecting tests in 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\AdHocConsoleRunner\bin\Debug\AdHocConsoleRunner.exe'
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery started
NSpec VS Adapter 0.1.0.0: [INFO]  Skipping binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\ConfigSampleSystem\bin\Debug\ConfigSampleSystem.exe' because it does not reference nspec library
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery finished
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery started
NSpec VS Adapter 0.1.0.0: [INFO]  Skipping binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSystem\bin\Debug\SampleSystem.dll' because it does not reference nspec library
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery finished
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery started
NSpec VS Adapter 0.1.0.0: [INFO]  Discovering tests in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [DEBUG] Start collecting tests in 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [DEBUG] Finish collecting tests in 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\AdHocConsoleRunner\bin\Debug\AdHocConsoleRunner.exe'
NSpec VS Adapter 0.1.0.0: [INFO]  Found 0 tests
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery finished
NSpec VS Adapter 0.1.0.0: [TRACE] Debug info found for method 'SampleSpecs.ParentSpec'.'<method_context_1>b__2_0' in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [TRACE] Debug info found for method 'SampleSpecs.ParentSpec'.'<method_context_1>b__2_1' in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [TRACE] Debug info found for method 'SampleSpecs.ParentSpec'.'<method_context_2>b__3_0' in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [DEBUG] Cannot get debug info for method 'NSpec.nspec'.'<.ctor>b__0' in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [TRACE] Debug info found for method 'SampleSpecs.ChildSpec'.'<method_context_4>b__1_0' in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [TRACE] Debug info found for method 'SampleSpecs.ChildSpec'.'<method_context_5>b__2_1' in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [TRACE] Debug info found for method 'SampleSpecs.ChildSpec'.'<method_context_5>b__2_2' in binary 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [DEBUG] Finish collecting tests in 'D:\WS\NET\NSpec framework\NSpec.VsAdapter\src\NSpec.VsAdapter\Test\Samples\SampleSpecs\bin\Debug\SampleSpecs.dll'
NSpec VS Adapter 0.1.0.0: [INFO]  Found 7 tests
NSpec VS Adapter 0.1.0.0: [INFO]  Discovery finished
========== Discover test finished: 0 found (0:00:00,4990949) ==========
BrainCrumbz commented 8 years ago

This happened while another adapter (NSpec Test Adapter) was also installed in VS2015. After uninstalling that and opening the same sample solution, tests are shown in Test Explorer window. It seems this issue can be closed for now.

amirrajan commented 8 years ago

May be worth exploring a discovery strategy where only method level contexts are includes (methods defined on a class) and not going any deeper than that.

BrainCrumbz commented 8 years ago

The discovery should enlist all found examples (in terms of NSpec domain), that are mapped to "runnable" test cases. What happens if one just stops at method level contexts? Maybe those have few or no examples at all defined, so in that case you'd see just a small part (or none) of available test cases.

amirrajan commented 8 years ago

True, but I've found I rarely run a single example. It's usually an entire context that I run. And I've found that deep nesting eventually evolves to class or method level contexts anyways.

Just something to consider (you can then just do static analysis of class structures).