search

nunit / nunit

NUnit Framework
https://nunit.org/
MIT License
2.51k stars 731 forks source link

Provide .NET Standard migration path from Framework 4.6.2 #2858

Closed mwpowellhtx closed 6 years ago

mwpowellhtx commented 6 years ago

Although I am displeased with the lack of WinForms/WPF migration path at the moment, I finally decided that it was time to begin migrating to .NET Standard. With either 2.6.4 or 2.6.5, I get this:

1> ... warning NU1701: Package 'NUnit 2.6.5' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.

This is causing a break in migration to .NET Standard 2.x and I am unable to move forward with my unit testing.

This will provide the seamless backwards compatible migration path for projects targeting Framework 4.6.2 or better.

Ironically, this might also be an opportune moment for me to reconsider my attitude towards Xunit, Combinatorial, etc.

Anyway, let me know what y'all decide to do re: Framework 4.6.2 and/or .NET Standard 2.0 (recommended).

CharliePoole commented 6 years ago

Is there a reason you have been sticking with NUnit V2? Maybe it's time to upgrade.

mwpowellhtx commented 6 years ago

If I read between the lines, are you saying v3 is on the .NET Standard migration path?

mwpowellhtx commented 6 years ago

Upgrading is simple enough. However, if I am reading the "results" accurately, the lack of visual feedback in the Test Explorer is a gaping hole. The tests do not report inconclusive, they just don't seem to report anything at all, apart from what looks like some diagnostic messaging in the Output window. Am I far off base with that assessment?

mwpowellhtx commented 6 years ago

Test explorer can see the test hierarchy I've established. However, the NUnit adapter claims there are no tests.

My base class has the TestFixtureAttribute applied. I've got TestAttribute, for instance, applied at various appropriate levels in the hierarchy, at a base level on an abstract test method, and on from there.

The output says this:

[5/14/2018 4:26:53 PM Informational] ------ Run test started ------
[5/14/2018 4:26:54 PM Informational] NUnit Adapter 3.10.0.21: Test execution started
[5/14/2018 4:26:54 PM Informational] Running all tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll
[5/14/2018 4:26:54 PM Warning] Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll
[5/14/2018 4:26:54 PM Warning] An exception occurred in the driver while loading tests.
[5/14/2018 4:26:54 PM Warning]    at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter)
[5/14/2018 4:26:54 PM Warning] Innerexception: System.IO.FileNotFoundException: Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified.
File name: 'nunit.framework'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

[5/14/2018 4:26:54 PM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[5/14/2018 4:26:54 PM Warning] No test is available in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[5/14/2018 4:26:54 PM Informational] ========== Run test finished: 0 run (0:00:01.5930911) ==========

But I don't buy it. This was working before no problems.

mwpowellhtx commented 6 years ago

I should take a step backward here and explain, I've got a suite of unit test "API" of sorts, which I want to extend into a specific set of unit tests. My goal is for users to be able to subscribe to these boilerplate unit tests along with the core assembly package in order to best verify their usage of that assembly. So, what I am aiming to do is package the NUnit as a dependency, bare minimum. I may have to include the adapter, but I doubt it, at least not for the underlying engine. With logging enabled, this is what I end up with:

[5/14/2018 4:48:56 PM Informational] ------ Run test started ------
[5/14/2018 4:48:57 PM Informational] NUnit Adapter 3.10.0.21: Test execution started
[5/14/2018 4:48:58 PM Informational] Running all tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll
[5/14/2018 4:48:58 PM Warning] Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll
[5/14/2018 4:48:58 PM Warning] An exception occurred in the driver while loading tests.
[5/14/2018 4:48:58 PM Warning]    at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter)
[5/14/2018 4:48:58 PM Warning] Innerexception: System.IO.FileNotFoundException: Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified.
File name: 'nunit.framework'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)

=== Pre-bind state information ===
LOG: DisplayName = nunit.framework
 (Partial)
WRN: Partial binding information was supplied for an assembly:
WRN: Assembly Name: nunit.framework | Domain ID: 4
WRN: A partial bind occurs when only part of the assembly display name is provided.
WRN: This might result in the binder loading an incorrect assembly.
WRN: It is recommended to provide a fully specified textual identity for the assembly,
WRN: that consists of the simple name, version, culture, and public key token.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue.
LOG: Appbase = file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/
LOG: Initial PrivatePath = NULL
Calling assembly : (Unknown).
===
LOG: This bind starts in default load context.
LOG: Configuration file G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll.config does not exist.
LOG: No application configuration file found.
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind).
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/nunit.framework.DLL.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/nunit.framework/nunit.framework.DLL.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/nunit.framework.EXE.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/nunit.framework/nunit.framework.EXE.

[5/14/2018 4:48:58 PM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[5/14/2018 4:48:58 PM Warning] No test is available in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[5/14/2018 4:48:58 PM Informational] ========== Run test finished: 0 run (0:00:01.6150923) ==========
CharliePoole commented 6 years ago

I'm saying that NUnit V2 is built to target .NET 2.0 (older versions 1.x as well) and can only run on compatible platforms. For .NET Standard you need NUnit 3. Unless something is blocking you, do that migration first.

On Mon, May 14, 2018, 9:49 PM Michael W Powell notifications@github.com wrote:

I should take a step backward here and explain, I've got a suite of unit test "API" of sorts, which I want to extend into a specific set of unit tests. My goal is for users to be able to subscribe to these boilerplate unit tests along with the core assembly package in order to best verify their usage of that assembly. So, what I am aiming to do is package the NUnit as a dependency, bare minimum. I may have to include the adapter, but I doubt it, at least not for the underlying engine. With logging enabled, this is what I end up with:

[5/14/2018 4:48:56 PM Informational] ------ Run test started ------ [5/14/2018 4:48:57 PM Informational] NUnit Adapter 3.10.0.21: Test execution started [5/14/2018 4:48:58 PM Informational] Running all tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll [5/14/2018 4:48:58 PM Warning] Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll [5/14/2018 4:48:58 PM Warning] An exception occurred in the driver while loading tests. [5/14/2018 4:48:58 PM Warning] at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage) at NUnit.Engine.Runners.DirectTestRunner.LoadPackage() at NUnit.Engine.Runners.TestDomainRunner.LoadPackage() at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded() at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter) at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter) at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter) [5/14/2018 4:48:58 PM Warning] Innerexception: System.IO.FileNotFoundException: Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified. File name: 'nunit.framework' at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks) at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks) at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks) at System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark) at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo) at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes) at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes) at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes) at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args) at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings) at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)

=== Pre-bind state information === LOG: DisplayName = nunit.framework (Partial) WRN: Partial binding information was supplied for an assembly: WRN: Assembly Name: nunit.framework | Domain ID: 4 WRN: A partial bind occurs when only part of the assembly display name is provided. WRN: This might result in the binder loading an incorrect assembly. WRN: It is recommended to provide a fully specified textual identity for the assembly, WRN: that consists of the simple name, version, culture, and public key token. WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue. LOG: Appbase = file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/ LOG: Initial PrivatePath = NULL Calling assembly : (Unknown).

LOG: This bind starts in default load context. LOG: Configuration file G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll.config does not exist. LOG: No application configuration file found. LOG: Using host configuration file: LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config. LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind). LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/nunit.framework.DLL. LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/nunit.framework/nunit.framework.DLL. LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/nunit.framework.EXE. LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Collections.Enumerations.Tests.Tests/bin/Debug/netstandard2.0/nunit.framework/nunit.framework.EXE.

[5/14/2018 4:48:58 PM Informational] NUnit Adapter 3.10.0.21: Test execution complete [5/14/2018 4:48:58 PM Warning] No test is available in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Collections.Enumerations.Tests.Tests\bin\Debug\netstandard2.0\Kingdom.Collections.Enumerations.Tests.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again. [5/14/2018 4:48:58 PM Informational] ========== Run test finished: 0 run (0:00:01.6150923) ==========

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nunit/nunit/issues/2858#issuecomment-388957954, or mute the thread https://github.com/notifications/unsubscribe-auth/ACjfhaiNKMh80COFHgw9uZPeDoCHmFr9ks5tye3bgaJpZM4T-OtG .

mwpowellhtx commented 6 years ago

I think your package is broken. Its dependencies are not rolling up correctly. I start with a simple test assembly, nothing elaborate. My test class is literally this, and that's it:

namespace Testing
{
    using NUnit.Framework;

    [TestFixture]
    public class TestTests
    {
        [Test]
        public void Verify()
        {
            Assert.That(1 + 1, Is.EqualTo(2));
            Assert.That(2 + 2, Is.EqualTo(4));
        }
    }
}

And the errors are similar:

[5/14/2018 7:42:30 PM Informational] ------ Run test started ------
[5/14/2018 7:42:31 PM Informational] NUnit Adapter 3.10.0.21: Test execution started
[5/14/2018 7:42:31 PM Informational] Running all tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Testing\bin\Debug\netstandard2.0\Kingdom.Testing.dll
[5/14/2018 7:42:31 PM Warning] Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Testing\bin\Debug\netstandard2.0\Kingdom.Testing.dll
[5/14/2018 7:42:31 PM Warning] An exception occurred in the driver while loading tests.
[5/14/2018 7:42:31 PM Warning]    at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter)
[5/14/2018 7:42:31 PM Warning] Innerexception: System.IO.FileNotFoundException: Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified.
File name: 'nunit.framework'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)

=== Pre-bind state information ===
LOG: DisplayName = nunit.framework
 (Partial)
WRN: Partial binding information was supplied for an assembly:
WRN: Assembly Name: nunit.framework | Domain ID: 4
WRN: A partial bind occurs when only part of the assembly display name is provided.
WRN: This might result in the binder loading an incorrect assembly.
WRN: It is recommended to provide a fully specified textual identity for the assembly,
WRN: that consists of the simple name, version, culture, and public key token.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue.
LOG: Appbase = file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/
LOG: Initial PrivatePath = NULL
Calling assembly : (Unknown).
===
LOG: This bind starts in default load context.
LOG: Configuration file G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Testing\bin\Debug\netstandard2.0\Kingdom.Testing.dll.config does not exist.
LOG: No application configuration file found.
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind).
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/nunit.framework.DLL.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/nunit.framework/nunit.framework.DLL.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/nunit.framework.EXE.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/nunit.framework/nunit.framework.EXE.

[5/14/2018 7:42:31 PM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[5/14/2018 7:42:31 PM Warning] No test is available in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Testing\bin\Debug\netstandard2.0\Kingdom.Testing.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[5/14/2018 7:42:31 PM Informational] ========== Run test finished: 0 run (0:00:01.626093) ==========
mwpowellhtx commented 6 years ago

@CharliePoole Can you be more specific about this:

Versions of ReSharper earlier than the 8.2 version has an issue with the NuGet adapter, which will prevent NUnit tests from running. Make sure you have updated Reshaper to at least version 8.2.

Version 8.2? That sounds pretty ancient, at least according to JetBrains versioning in the past couple of years.

mwpowellhtx commented 6 years ago

@CharliePoole I upgraded from 2017.x to 2018.x just now; current, 2018.1, I think. I am getting errors that tell me I cannot run NUnit tests on .NET Standard applications. Must be compiled as .NET Core Application.

What gives? Same sort of errors as before. All I know is, testing under 2.6.x works just fine.

[5/14/2018 10:06:32 PM Informational] ------ Run test started ------
[5/14/2018 10:06:33 PM Informational] NUnit Adapter 3.10.0.21: Test execution started
[5/14/2018 10:06:33 PM Informational] Running all tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Testing\bin\Debug\netstandard2.0\Kingdom.Testing.dll
[5/14/2018 10:06:34 PM Warning] Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Testing\bin\Debug\netstandard2.0\Kingdom.Testing.dll
[5/14/2018 10:06:34 PM Warning] An exception occurred in the driver while loading tests.
[5/14/2018 10:06:34 PM Warning]    at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter)
[5/14/2018 10:06:34 PM Warning] Innerexception: System.IO.FileNotFoundException: Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified.
File name: 'nunit.framework'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)

=== Pre-bind state information ===
LOG: DisplayName = nunit.framework
 (Partial)
WRN: Partial binding information was supplied for an assembly:
WRN: Assembly Name: nunit.framework | Domain ID: 4
WRN: A partial bind occurs when only part of the assembly display name is provided.
WRN: This might result in the binder loading an incorrect assembly.
WRN: It is recommended to provide a fully specified textual identity for the assembly,
WRN: that consists of the simple name, version, culture, and public key token.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue.
LOG: Appbase = file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/
LOG: Initial PrivatePath = NULL
Calling assembly : (Unknown).
===
LOG: This bind starts in default load context.
LOG: Configuration file G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Testing\bin\Debug\netstandard2.0\Kingdom.Testing.dll.config does not exist.
LOG: No application configuration file found.
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind).
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/nunit.framework.DLL.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/nunit.framework/nunit.framework.DLL.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/nunit.framework.EXE.
LOG: Attempting download of new URL file:///G:/Source/Kingdom Software/Kingdom.Collections/Migrate/src/Kingdom.Testing/bin/Debug/netstandard2.0/nunit.framework/nunit.framework.EXE.

[5/14/2018 10:06:34 PM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[5/14/2018 10:06:34 PM Warning] No test is available in G:\Source\Kingdom Software\Kingdom.Collections\Migrate\src\Kingdom.Testing\bin\Debug\netstandard2.0\Kingdom.Testing.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[5/14/2018 10:06:34 PM Informational] ========== Run test finished: 0 run (0:00:01.6330934) ==========
CharliePoole commented 6 years ago

@mwpowellhtx the quotes sentence in the docs refers to extremely old versions of Resharper. You do not appear to be running tests with resharper as far as I can see.

Please tell us the exact versions of all the nunit packages you are using. Your log indicates version 3.10 of the NUnit 3 Visual Studio Test Adapter but you earlier wrote that you are using the NUnit framework 2.6.5 and 2.6.4, which makes no sense as a combination. What other runners are you using, if any?

mwpowellhtx commented 6 years ago

@CharliePoole No, you have that wrong. I'm not using them in combination. I have a couple of test assemblies which run fine via Visual Test Adapter. However, attempts to run even the simplest of tests using the latest NUnit, or Xunit for that matter, are now met with claims that I can only test in .NET Core. But this makes no sense as my assembly(ies) are targeting .NET Standard. Yes, the errors are Output diagnostic reports it would seem, when I try to run in the Test Explorer, at least. R# 2018.1 doesn't "see" the unit tests as test, but instead, I gather, is prohibiting the issue over claims of Standard vs Core. Really kind of confused by the whole mess at this point.

CharliePoole commented 6 years ago

All test assemblies must target some platform rather than just targeting .NET Standard. For testing libraries that target some version of .NET Standard, .NET Core is frequently used.

As far as I know the requirement for test assemblies to target a platform is something we are stuck with.

mwpowellhtx commented 6 years ago

@CharliePoole I don't know what that means. I can't just "fall back on" .NET Standard? What's the point of it being "standard" then if I have to chip away at the edges just to get something to work correctly?

Which, taking the circumspect look at it, is much of my confusion re: the whole .NET refactoring business. Neither here nor there solving the issue at the moment outside perhaps informing my next set of questions, to whom, etc.

Thanks for any further insights you can offer.

CharliePoole commented 6 years ago

This is not something under the control of nunit or xunit for that matter.

mwpowellhtx commented 6 years ago

@CharliePoole I found an issue which discussed it in some depth. Glanced at it this evening, but I will chew on it in more depth tomorrow. Just in summary, not a runtime but an "API surface area". Whatever the heck that means... Why offer it as a target option at all then? Makes no sense to me. Never mind whether I want to subscribe to daily builds, quarterly, or what have you. Having a hard time selling me on the run time at this point. Anyway, if I encounter anything more specific I will follow up with those issues accordingly.

mwpowellhtx commented 6 years ago

@CharliePoole I get it. We're all kind of at the mercy of you know who.

mwpowellhtx commented 6 years ago

Sorted it. My test assemblies now target Framework 4.6.2, at least for the moment. Thanks for the .NET Standard clue. I had no idea, as it seems is the case for many unsuspecting .NET aficionados out there, and I've never really dealt that hardcore with PCL or that whole migration path into .NET Standard, either. Mainly desktop.

jnm2 commented 6 years ago

Also see https://github.com/Microsoft/msbuild/issues/1310#issuecomment-346140547. The tooling team is aware of the need for .NET Standard test assemblies and wants to implement support, such as a <DeploymentFrameworks> csproj property. It's sadly just not in as much demand as other tooling features for the size of the team they have.

mwpowellhtx commented 6 years ago

@jnm2 Good to know, thanks for that tidbit.