search

nunit / nunit-console

NUnit Console runner and test engine
MIT License
214 stars 151 forks source link

NUnit Test Server #167

Open CharliePoole opened 7 years ago

CharliePoole commented 7 years ago

@CharliePoole commented on Sat Jun 20 2015

An Test Server would run continuously on the local or a remote machine. It would accept and queue requests to run tests and report on their status back to the requestor.

While it might seem to do the same thing as nunit-agent, it's a bit different:

  1. It's long running.
  2. It's a server. (NUnit-Agent actually works as a client, even though it performs a service!)
  3. Execution may be delayed so it needs to provide calls for the client to check status.
  4. We will need to have some sort of admin client to run it.
  5. We will need to have some way of centralizing the information about what servers are available.

We had a rudimentary server at one point in the life of NUnit 2.x. It shared a base class with nunit-agent. We may be able to reuse or re-purpose some of that code.

This is a post-3.0 feature.

@mklemarczyk commented on Tue Jun 23 2015

+1 I can provide some help with this issue in the future.

@ChristianPejrup commented on Thu Jun 25 2015

Very interesting idea, we have been talking about doing something similar in my team. We need some test grid (for lack of a better name) where we can execute long running integration tests. if there are some concrete tasks I would be happy to lend a hand :-)

@CharliePoole commented on Thu Jun 25 2015

I doubt this is something we will get into the 3.0 release. More to the point, we wouldn't want to delay the 3.0 release in order to get this feature in.

That said, I can go ahead and restore the old server code in order to let someone else work on it. Give me a few days.

@ChristianPejrup commented on Thu Jun 25 2015

That's all right, right now i'm personally more excited about seeing the final release of NUnit 3...this is just a cool feature. Until a server exist it should still be possible to do some RPC on another computer and running NUnit.exe commandline.

@oznetmaster commented on Sun Sep 20 2015

This is basically what I am doing in a small way with my CF builds. "Server" runs on CF device, and queues request from PC to run tests on the device (which run independently from the server).

@CharliePoole commented on Wed Jan 25 2017

This issue was written back when the engine, console and framework were all one project. Its actually an engine feature, so I'm moving it to the nunit-console project, which includes the engine.

JustinRChou commented 7 years ago

Isn't this only possible if the client and server are on the same network? Or is the client able to transfer the test dll to the server?

Never mind, thought it was going to be like a web server. But would there be such a case when you would want to send the dll over instead of the network path?

CharliePoole commented 7 years ago

All sorts of ifs and and buts exist and would have to be taken care of in the design. I'd say let's wait till we can actually deal with it. IMO, the Engine is not yet mature enough to stretch the extra distance of keeping track of available servers, etc.

I think there are three general options for how the test assembly gets on the server:

  1. It's already there - staged using some other software than NUnit
  2. It's on a share that can be accessed by the server
  3. The dll is sent there by NUnit. I'm not really ready to do anything with this, but I was answering a question on the gui project and came across the orphaned issue. I wanted to get it to the right location.
stevenaw commented 1 year ago

As noted in discussion

@VentuzKarolStrozynski has provided a great example and use case for why this is a desirable feature

I would use the NUnit Console and it could be on both Jenkins and the Remote Tester. But in the NUnit Console command line options I do not see how to specify where the tests should be executed. To give you a bigger picture: I'm looking for a replacement for MSTest.exe and the VS Test Controller/Agent collaboration which allowed to trigger a unit test on machine A but the execution would take place on machine B. It seems that this is not possible with .net 6 anymore respectively does MSTest V2 not support remote test execution. I need this remote execution because these tests are pretty GPU intensive and need huge resources located on the tester are required. Additionally I need to test on a machine without any Visual Studio installations.