search

nunit / nunit-vs-adapter

Runs NUnit V2 tests inside the Visual Studio 2012 or later Test Explorer window.
MIT License
50 stars 43 forks source link

Nunit 2 test adapter does not support Visual Studio 2019 #180

Closed bkilada closed 5 years ago

bkilada commented 5 years ago

The NUnit 2 test adapter doesn't support Visual Studio 2019, which was released a few days ago. Are there any plans implement support?

Thanks, Ben

OsirisTerje commented 5 years ago

It is supported by the nuget adapter, but not the vsix. The number of vsix downloads are very low now, so , yes, we can update it, it doesn't take much, but is it really needed ?

As mentioned other places, we would really like to see people moving to the nuget adapter, and support for vsix will be deprecated in the VS anyway going forward.

n3wjack commented 5 years ago

For some reason the nuget adapter isn't always enough. It stopped working for me in 2018 at some point. Running all tests is OK, but if I want to run a specific test, it doesn't or run them. It can't discover them for some reason according to the output window. So I resorted to the vsix, which fixed that issue in 2018. In 2019 I now have the same issue, but no vsix to fix that with. :(

bkilada commented 5 years ago

Ah, I didn't know there is a NuGet adapter. I tried that but unfortunately ran into similar issues to @n3wjack. I run a test and it immediately fails with no error message. The NUnit 3 NuGet adapter had the same problem, only the NUnit 3 extension worked.

n3wjack commented 5 years ago

It's a weird problem. In the VS test explorer the tests stay blue as in "not run tests". So the tests are discovered, but when you select one or more, or use the shortcut keys to run the tests in the current open file, the test run starts but doesn't find any tests. The funny thing is that when you use "run all tests" everything works.

OsirisTerje commented 5 years ago

Can you upload a small repro?

n3wjack commented 5 years ago

This behaves the same. It works in VS2017, with the vsix test adapter installed. In VS2019, it doesn't run the tests, but it does discover them.

I installed the nuget packages in this order:

nunit2_testadapter_vs2019.zip

OsirisTerje commented 5 years ago

@n3wjack

  1. The vsix doesn't work with VS 2019. We'll try to get out a new release with that supported shortly.
  2. You have included the nuget adapter, but the vsix takes precendence.
    If you uninstall the vsix, then the nuget adapter should work. Can you check if all works then? (3. The discovery you see is probably not from the adapter, but from the source based real time discovery in the test explorer. )
n3wjack commented 5 years ago

I know the VSIX doesn't work in 2019, it's not in the plugin list, which is why I ended up here. :)

I now disabled the VSIX in VS2017 and now I have the same behavior there as in VS2019. Tests are discovered (you see them in the test explorer), but aren't run when I want to run an individual test or use "Run all". This is exactly why I installed the VSIX in 2017.

OsirisTerje commented 5 years ago

@n3wjack Thanks for the repo and the explanations! I can confirm the bug, a bit surprised about it in fact.
I'll come back to you on this as soon as possible.
Thanks again, really appreciate this !

n3wjack commented 5 years ago

Glad it's reproducible. That makes it a lot easier to fix it. Good luck!

Op ma 15 apr. 2019 19:06 schreef Terje Sandstrom notifications@github.com:

@n3wjack https://github.com/n3wjack Thanks for the repo and the explanations! I can confirm the bug, a bit surprised about it in fact. I'll come back to you on this as soon as possible. Thanks again, really appreciate this !

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nunit/nunit-vs-adapter/issues/180#issuecomment-483337065, or mute the thread https://github.com/notifications/unsubscribe-auth/ACJYEokboWumajFV1YIm-0dbnTvoD61Yks5vhLGBgaJpZM4cfeJt .

OsirisTerje commented 5 years ago

UPDATE: Visual Studio should manage to discover the adapters when they are in the packages folder, but it seems there is some change there. You mention it stopped working "some time" in 2018, and I think that was a minor update change.

In the options for Test it says:

image

(2) above explains why you get it working with the vsix, which currently doesn't help you with VS2019. (1) above explains why it don't find it, because the 2.1 adapter doesn't copy itself to the binary output of the test project, which it should do.

I am pretty sure that before some time in 2018, VS managed to find them under the packages folder, but that doesn't help much now.

WORKAROUND: What you can do, is to add a runsettings file: image where you set the adapter path to some local folder where you have manually copied the adapter dll's: image

When I do that with your repo, it starts working again: image

This is only a workaround, but should work both locally and on a build server.

We need to fix this in the adapter too. The following can be done: 1) Update the vsix to support installation in VS2019. I think we should do this. 2) Add the copying of the nuget adapter dlls to the test project outputs the same way as it is done in the NUnit3 adapter. I think we should do this too.

OsirisTerje commented 5 years ago

@n3wjack @bkilada The pre-release package for the 2.2 adapter is now available on https://www.myget.org/feed/nunit/package/nuget/NUnitTestAdapter If you could check it out, it would be awesome.

bkilada commented 5 years ago

Awesome. No promises but I’ll try if I get a chance.

From: Terje Sandstrom notifications@github.com Sent: Wednesday, June 5, 2019 6:59:04 AM To: nunit/nunit-vs-adapter Cc: bkilada; Mention Subject: Re: [nunit/nunit-vs-adapter] Nunit 2 test adapter does not support Visual Studio 2019 (#180)

@n3wjackhttps://github.com/n3wjack @bkiladahttps://github.com/bkilada The pre-release package for the 2.2 adapter is now available on https://www.myget.org/feed/nunit/package/nuget/NUnitTestAdapter If you could check it out, it would be awesome.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/nunit/nunit-vs-adapter/issues/180?email_source=notifications&email_token=AFUXYD5BICPWRBIDRY6XDZ3PY7BCRA5CNFSM4HD54JW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW7ZJHQ#issuecomment-499094686, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFUXYD2G7WJJVCM3S35HHJLPY7BCRANCNFSM4HD54JWQ.

OsirisTerje commented 5 years ago

@bkilada It is now out on nuget, as version 2.2.0, so it is released :-) If you find anything, give a ping. :-)

n3wjack commented 5 years ago

Just upgraded the package in our solution and the unittests are now discovered and run in VS2019. 👍 Thanks for fixing this!

OsirisTerje commented 5 years ago

@n3wjack Cool! Thanks !

ValMetcalf commented 4 years ago

I still have the exact behaviour as reported by n3wjack.

I'm in Visual Studio 2019 and have the NUnit3TestAdapter (3.16.1) installed. I get the warning NU1701 about that package not being compatible with my project. The tests are dscovered, but not run. Which test adapter should I be using?

mikkelbu commented 4 years ago

It depends on which version of NUnit framework you are using for your test.

ValMetcalf commented 4 years ago

I am using NUnit3 and NUnit3TestAdapter so I'd expect it to work.

mikkelbu commented 4 years ago

Ok. So this is not the same problem as n3wjack as the original issue was about NUnit 2 and NUnitTestAdapter. Some questions:

image

Mykhailo-Seniutovych commented 4 years ago

I think it's not fair to stop supporting the extension and impose using NuGet package only. For example, in my team some people use Resharper for running unit tests, some people might use other tools to run the tests. I would like to try to use the NUnitTestAdapter but I don't want to add a NUnitTestAdapter NuGet package to the solution, and "impose" this tool on other people. I think you should provide a tool that can work independently for each individual developer.