Design and add support for NUnit engine extensions

[CharliePoole]

Although we recognize NUnit extension attributes and will try to load them if they are present, loading of existing extensions does not succeed in most cases. The reasons for this vary from one extension to another, so each extension needs to be addressed separately. In general, the problem is that the existing extensions implement NUnit interfaces whereas TestCentric is expecting a TC interface of the same name. In addition, some extension methods return objects defined within NUnit, which we do not support.

We could solve these problems easily by importing and directly using the NUnit types but this would only be useful for extension points for which we do not expect to have TC-specific extensions. If we intend to have both NUnit and TC extensions for a given extension point, then some way for the two to co-exist must be developed.

Some extensions may be developed by third-parties, including NUnit users. Since we will usually be unaware of such extensions, they are not included in the scope of this issue, with the exception of ITestEventListener. We know that has been implemented by many users and will need to be completely supported for that reason

The scope of this issue includes the following ExtensionPoints, which exist in NUnit and are duplicated in TestCentric:

• IDriverFactory (Existing extension: nunit-v2-framework-driver)
• IProjectLoader (Existing extensions: nunit-project-loader and vs-project-loader)
• IResultWriter (Existing extension: nunit-v2-result-writer)
• IService (No existing extensions)
• ITestEventListener (Existing extension: teamcity-event-listener)

[CharliePoole]

As a first step towards design, we should consider whether we wish to support each of the listed NUnit extensions in TestCentric. The following is a first cut, which may be updated as we proceed.

IDriverFactory

The existing V2 Driver must be supported. Support for other extensions, if they exist, is not necessary.

IProjectLoader

The existing NUnit project loader must be supported. The Visual Studio project loader is extremely limited and we could replace it with a TestCentric version. Optionally, both could be internalized.

IResultWriter [DONE]

The existing V2 Result Writer should be supported, since so many third-party apps still require the V2 format.

IService [NOT NEEDED]

Since there are no existing NUnit extensions and (to my knowledge) none under development, this does not need to be supported. The TC extension point will be retained for future development

ITestEventListener [DONE]

This needs strong support because of it's extensive use by users. In addition, we should support the existing Team City event listener.

[CharliePoole]

Implementation Notes

• After some initial experimentation with modification of TestCentric.Engine.Api, I decided to do leave the api as it is, clean of NUnit references. Instead, I will use wrappers at the point where we collect extensions for each extension point. Hopefully, these will be the only places where we need to use interfaces from NUnit.Engine.Api.

• PR #200 creates a wrapper for NUnit.Engine.ITestEventHandler and update TestEventDispatcher to use it. This was the simplest interface with which to start, as it has only one method and it returns void.

• The next interface in order of difficulty is IResultWriter, with three methods returning void. It adds complexity because it has a property indicating the format supported by the writer. This requires us to deal with the ExtensionNode in addition to the actual interface. Completed in PR #201.

• IProjectLoader can be wrapped... but it's much more complicated. Two of the methods return NUnit.Engine.Extensibility.IProject, which would also need wrapping. Doing so would require us to implement each project class (e.g. NUnitProject) and the project classes are defined in the NUnit extensions themselves. In addition, each project class makes use of NUnit.Engine.TestPackage and that will have to be wrapped. The ability to change the active configuration depends on having direct access to each project class as well. So... a different approach is preferable.

• For NUnitProjectLoader I'm going to try an "internal extension", similar to what we do for the built-in result writers.

• For VSProjectLoder I'm still unsure.