search

nunit / nunit3-vs-adapter

NUnit 3.0 Visual Studio test adapter for use under VS 2012 or later
https://nunit.org
MIT License
205 stars 106 forks source link

TestPlatformException error discovering tests in solution with netstandard extensions library #656

Open toddlindsey opened 5 years ago

toddlindsey commented 5 years ago

VS2019 16.2.4 NUnit 3.12.0 NUnit3TestAdapter 3.15.1 TestSDK 16.2.0

Start a new solution with a .NET Core (or Net Framework, I don't think it or version matters) project (with nunit+adapter+sdk references) referencing a netstandard2.0 project that contains only the nunit package reference, and build.

Test Discovery phase generates error: Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Unable to find ... NetstandardLib\bin\Debug\netstandard2.0\testhost.dll. Please publish your test project and retry.

The issue here is I need the ability to create various types of re-usable assets that require the NUnit project reference in class libraries that are not themselves test projects. This can be any number of things, including custom NUnit attributes that are applied to tests. It seems the mere presence of the NUnit reference is forcing a test discovery and thus requiring the adapter, which isn't appropriate for a netstandard project (and doesn't work anyway). I also tried setting IsTestProject to false in the csproj, but this did not help.

Let me know if I'm missing something here...

OsirisTerje commented 5 years ago

Can you please upload a repro project? Make it as small as you can, it should compile "as is".

Without the adapter it should not even try to discover them.

toddlindsey commented 5 years ago

https://github.com/toddlindsey/NUnitAdapterIssue656

OsirisTerje commented 5 years ago

(Sorry, notifications stopped coming for some reason. I'll have a look)

OsirisTerje commented 5 years ago

When I run the tests, I see: image But in the console output: image

Since VS is using real time discovery, the standard NUnit discoveryphase is really not needed here (except parts of it is running during execution), but it should in any way not give this response.

And just to be clear, the path is as you described above, to the StandardLib folder: image And, it looks for the testhost.dll, so this is somehow related to how dotnet test works. I'll also contact the PG about this.

@toddlindsey Is what I describe here consistent with what you see ?

OsirisTerje commented 5 years ago

If I add some code to the library and use that in the test, it still works. So I don't think this is a blocker for you. The error message should ofc be fixed in any case. image

The SomeCoolStuff method just returns the parameter, but it is INSIDE the netstandard project, which should be the important thing there.

toddlindsey commented 5 years ago

Yes the tests run, so not a blocker but following up because of the noise and concern this could be related to discovery times on a very large solution I have to work with, or performing more discoveries than needed because the last one failed.

It seems the real-time discovery works great though; not sure if there is a way to turn off the separate discovery than runs before the tests start?

Jetski5822 commented 2 years ago

Im getting the same issue - where IsTestProject is just ignored.

OsirisTerje commented 2 years ago

@Jetski5822 Not sure I understand what is your issue. Is it the same as above, or do you have a different issue ?

Jetski5822 commented 2 years ago

Ahh yeah, I wasnt really clear @OsirisTerje let me add some details. In my solution, I have a number of projects that have NUnit dependencies to extend the existing functionality of NUnit, but those projects themselves are not test projects. So, I mark these projects with 

<IsTestProject>false</IsTestProject>

The issue here is that these projects still show one of three errors when discovering tests

First:

Test project SomeCompany.UnitTests.Data.Oracle.Common does not reference any .NET NuGet adapter. Test discovery or execution might not work for this project.
It's recommended to reference NuGet test adapters in each test project in the solution.

Second

No test is available in C:\code\disco\dotnet-shared\src\SomeCompany.Components\SomeCompany.Components.Planning.Test.Utilities\bin\Debug\net5.0\SomeCompany.Components.Planning.Test.Utilities.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Third

Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Unable to find ... NetstandardLib\bin\Debug\netstandard2.0\testhost.dll
^^ Same as the original poster

Just because you have NUnit referenced doesnt mean there are tests in the project; especially given that we are overriding the IsTestProject attribute from the Test.Sdk.

Thoughts?