search

nunit / nunit-vs-adapter

Runs NUnit V2 tests inside the Visual Studio 2012 or later Test Explorer window.
MIT License
50 stars 43 forks source link

vstest.console.exe TestCaseFilter does not work in 3.16 when the project has no tests that match the filter #187

Closed tiwilson7 closed 4 years ago

tiwilson7 commented 4 years ago

Problem

When running tests via vstest.console.exe using TestCaseFilter to filter based on TestCategory, the filter is not correctly applied when the tested assembly doesn't contain any tests that match the filter.

The filter is correctly applied if one or more tests in the tested assembly match the filter.

This is a breaking change in 3.16. I confirmed that this works correctly in 3.15.1.

Steps to recreate

The batch file executes vstest.console.exe against four test projects using a test category filter. Two of the test projects use NUnit3TestAdapter 3.15.1 and two of them use 3.16.

One project of each adapter version contains one test with a test category that matches the test category filter and the other contains only tests with no test category.

Expected

No tests are executed in the NoTestCategory315 and NoTestCategory316 projects.

Actual

All tests are (incorrectly) executed in the NoTestCategory316 project even though no test matches the filter.

NoTestCategory315 does not execute any tests as expected.

mikkelbu commented 4 years ago

This sounds like a duplicate of nunit/nunit3-vs-adapter#686.

Ps. note that this repository is for the nunit adapter for NUnit 2.

tiwilson7 commented 4 years ago

Thanks @mikkelbu. Closing.