nunit / nunit3-vs-adapter

NUnit 3.0 Visual Studio test adapter for use under VS 2012 or later
https://nunit.org
MIT License
205 stars 106 forks source link

Explicit test is running, but it shouldn't #1223

Open ossysek opened 1 week ago

ossysek commented 1 week ago

Hello,

I wrote one simple test and it should not be run, but it started:

image

My code:

using NUnit.Framework;

namespace NunitExplicit.Test
{
    [TestFixture]
    public class UnitTest1
    {
        [Test]
        [Explicit]
        public void TestMethod1()
        {
            Assert.True(true);
        }
    }
}

My dependencies: image

It seems, there is a bug, because when I add second test without explicit, everything start to work correctly: image

using NUnit.Framework;

namespace NunitExplicit.Test
{
    [TestFixture]
    public class UnitTest1
    {
        [Test]
        [Explicit]
        public void TestMethod1()
        {
            Assert.True(true);
        }

        [Test]
        public void TestMethod2()
        {
            Assert.True(true);
        }
    }
}

Next test: image

using NUnit.Framework;

namespace NunitExplicit.Test
{
    [TestFixture]
    [Explicit]
    public class UnitTest1
    {
        [Test]
        [Explicit]
        public void TestMethod1()
        {
            Assert.True(true);
        }

        [Test]
        public void TestMethod2()
        {
            Assert.True(true);
        }
    }
}

According to the dokumentation, it shouldn't work like that.

Microsoft Visual Studio Enterprise 2022 (64-bit) - Current - Version 17.10.4 Target freamwork: .NET 8.0

manfred-brands commented 1 week ago

@ossysek, how are you running your tests?

If I run dotnet test from the command line, I get:

$ dotnet test --no-build
NUnit Adapter 4.5.0.0: Test execution started
Running all tests in D:\Development\Playground\Explicit\bin\Debug\net9.0\Explicit.dll
   NUnit3TestExecutor discovered 1 of 1 NUnit test cases using Current Discovery mode, Explicit run
NUnit Adapter 4.5.0.0: Test execution complete
  Explicit test succeeded (0.7s)

Test summary: total: 1, failed: 0, succeeded: 0, skipped: 0, duration: 0.7s

When I remove the [Explicit], the result shows:

Test summary: total: 1, failed: 0, succeeded: 1, skipped: 0, duration: 0.7s

However, If I run the tests in Visual Studio by right click on project and selecting 'Run tests', it runs regardless of attribute. Maybe it is a feature that if there is only one test and you select 'Run tests' that it runs that test. But it seems to be one test in a fixture as even when I have 3 classes, each with 1 Explicit test they run.

@OsirisTerje Not sure if this is a feature of VS or the adapter.

ossysek commented 5 days ago

@manfred-brands thank you for your reply.

I run the tests in two ways:

  1. Right mouse click in Test Explorer and "Run"
  2. Run whole project in Azure Pipeline

In Pipeline the test are run as well.