nunit / nunit-console

NUnit Console runner and test engine
MIT License
215 stars 151 forks source link

Multicomputer/Multiplatform testing environment #944

Open oznetmaster opened 3 years ago

oznetmaster commented 3 years ago

I have been supporting the use of NUnit in a two computer testing environment for years now.

That environment has been a Windows/.NET 4.7 PC Host, and a Windows CE/Compact Framework 3.5 Test machine, connected by SSH and SFTP.

I run a GUI on the Host side, and NUnitLite on the test machine.

I currently support about 99% of the full NUnit 3.13.2 testing framework in that environment,

My context is now changing. I now need to run a Windows/.NET Core PC Host machine, and a Linux/Mono/.NET 4.6 Test machine.

Since I now have access to the full Linux/Mono environment, I would like to be able to run a more normal set of testing pieces. My preference would be a NUnit Console or TestCentric GUI on the Host machine, and normal test assemblies (not NUnitLite) on the Test machine.

It is unclear to me from all of the discussions on both this and the TestCentric repo how close to being able to do this things are.

I am currently using the exact same testing environment/communication protocols as I was previously on the Host machine, and using unmodified mono NUnitLite on the Test machine, controlling the testing through the command line on the test machine as was previously the case. This works, but limits things to the way they have always been. None of the capabilities of the engine/agent, or the new GUI, are available, and all test assemblies must be compiled with NUnitLite.

Where are things in regards to allowing the full environment I would like to make available working? This seems to need a full TCP based engine/agent communications protocol, and a fully supported cross platform (Windows & Linux/Mono) capability.

Since this is the environment I work in daily, I would like to be able to contribute to whatever work still needs to be done to make this possible.

I hope that @CharliePoole and @ChrisMaddock will be able to provide insight into this.

CharliePoole commented 3 years ago

Hi @oznetmaster

Brief answers as to current status from my point of view...

The TestCentric GUI has been running .NET Core tests using TCP for almost a year. It uses it's own engine, of course.

@ChrisMaddock is reviewing my PR #937 to enable doing the same with the NUnit Engine.

Both engines only support processes on the same machine, started by the engine itself. There's no capability built in to run tests on separate machines or devices and no ability to connect with pre-existing test servers.

The TestCentric engine supports pluggable agents - extensions that create an agent for running tests. NUnit has an issue to eventually do the same. My thinking is that one could write a pluggable agent that knew how to run on a device and install it.

If the TestCentric approach is of interest to you, we can discuss it separately. I'm looking for help. For obvious reasons :smile: this isn't the place to go into depth.

oznetmaster commented 3 years ago

@CharliePoole I emailed you offline. Did you get it? Has your email changed? Mine has not.

CharliePoole commented 3 years ago

Hi Neil,

I did get your note but I've been tied up with something else all day long. Now I'm going to bed, so... tomorrow!

Charlie

On Tue, May 4, 2021 at 12:27 AM Neil Colvin @.***> wrote:

@CharliePoole https://github.com/CharliePoole I emailed you offline. Did you get it? Has your email changed? Mine has not.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nunit/nunit-console/issues/944#issuecomment-831737957, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAUN7BLK3GXCGFI32HMHF4DTL6OWXANCNFSM44AKYGMQ .

CharliePoole commented 2 years ago

This seems like too big of a bite for 4.0. Perhaps 4.1.