tomaszeman / Xunit.Extensions.Ordering

Xunit extension with full support for ordered testing at collection, class and test case levels. Full-featured AssemblyFixture implementation.
Apache License 2.0
53 stars 9 forks source link

Fixture init failure marks unrelated tests as failed #4

Open oskarm93 opened 4 years ago

oskarm93 commented 4 years ago

Describe the bug When AssemblyFixture fails to initialize, it will fail all tests, including ones that did not implement from IAssemblyFixture for that type.

PS C:\workspace\AML\src\XUnitTestProject1> dotnet test
Test run for C:\workspace\AML\src\XUnitTestProject1\bin\Debug\netcoreapp2.2\XUnitTestProject1.dll(.NETCoreApp,Version=v2.2)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:00.69]     XUnitTestProject1.TestClass2.Test1 [FAIL]
[xUnit.net 00:00:00.70]     XUnitTestProject1.TestClass1.Test1 [FAIL]
  X XUnitTestProject1.TestClass2.Test1 [1ms]
  Error Message:
   System.Exception : hello world!
  Stack Trace:
     at XUnitTestProject1.AssemblyFixture2.InitializeAsync() in C:\workspace\AML\src\XUnitTestProject1\AssemblyFixture2.cs:line 11
  X XUnitTestProject1.TestClass1.Test1 [1ms]
  Error Message:
   System.Exception : hello world!
  Stack Trace:
     at XUnitTestProject1.AssemblyFixture2.InitializeAsync() in C:\workspace\AML\src\XUnitTestProject1\AssemblyFixture2.cs:line 11

Test Run Failed.
Total tests: 2
     Failed: 2
 Total time: 1.8082 Seconds

To Reproduce Steps to reproduce the behavior:

  1. See sample project here: XUnitTestProject1.zip
  2. Run dotnet test against this project.
  3. All tests will fail due to AssemblyFixture2 throwing an initialization exception.

Expected behavior To only fail TestClass2.Test1, and show TestClass1.Test1 as successful.

Desktop (please complete the following information):