Add a way to specify a custom resultwriter in .runsettings w/ TestOutputXml

[ranax404]

• NUnit: 3.12.0, NUnit3TestAdapter: 3.15.1
• VS2019, Version 16.3.6
• .net: Framework v4.8

I'm logging results by using a .runsettings with the TestOutputXml node. This works well, but on every run, the last result file is overwritten. The same behaves with the ConsoleRunner and param "--result". Therefore I created a custom resultwriter (NUnit.Extension.CustomResultWriter), which creates an additional result*.xml for each test and run (added a postfix and counter). This also works well with "--result=Result.xml";format=myformat for the ConsoleRunner.

The NUnit3TestExecutor calls the default ResultWriter "nunit3". I'm looking for a way to 1.) specify my own ResultWriter, 2.) to split the testresults into single files or 3.) append a postfix to the result.xml on each run (to not overwrite the last file)

[OsirisTerje]

@RoPe311 Good idea!
Do you have the source for your resultwriter - it is good to see the usage.

[ranax404]

@OsirisTerje Here's a simple example of my last custom resultwriter, which logs only relevant information for me in a complete result file and single testcase files. Single files are written as "testname+number.xml", additionally to my slimmed down result file.

For the NUnit3TestAdapter, maybe we can add a string property (e.g. "TestOutputFormat", default value = "nunit3") to the AdapterSettings class, which is then given as parameter to the following call in the NUnit3TestExecutor class: var resultWriter = resultService.GetResultWriter("nunit3", null);

CustomResultWriter.cs.txt

[ChrisMaddock]

Hi @OsirisTerje,

Was just coming to clarify, but looks like @RoPe311 has beaten me to it! All that I think is necessary is allowing the parameterization of the string nunit3 on this line here:

https://github.com/nunit/nunit3-vs-adapter/blob/1c4fbca7e31b898dd1297e58363fac8884fd8b42/src/NUnitTestAdapter/NUnit3TestExecutor.cs#L357

I think this would be a good approach to solving @RoPe311's problem - it'll make the best use of the existing adapter/engine functionality, without needing to add too much in the way of new features. :-)

[netcorefactory]

Any progress on this issue? In Azure Devops we have the rerun option to allow retest of flaky tests (Specflow/Selenium). This currently overwrites the xml file with the rerun results. Since we need this for QA reporting its quite a showstopper. Tips would be appreciated. Best solution would be a merge of results. Or at least write multiple files we could merge later.

[OsirisTerje]

@netcorefactory The fix was in, but didn't work due to the way the engine was integrated. @ChrisMaddock : We couldnt use the services at that time. Has things changed now with the new engine, so we can do this now?

[ChrisMaddock]

Hi Terje - looks like we fixed the issue with services in #719? It looks like you've written the code for this feature in https://github.com/nunit/nunit3-vs-adapter/commit/09afe86bee6b1645b2f6b8fb2f38699a01c119ab, but I'm not sure that commit was ever merged into master?

Let me know if I'm missing something here, it was a while ago we last looked at this one. 🙂

[OsirisTerje]

@netcorefactory I don't think you have and pretty sure you cant. There is nothing on the public repo here saying anything got in. So you're safe :-)

[netcorefactory]

your right been a while since using github :)

[netcorefactory]

Allright created a PR: https://github.com/nunit/nunit3-vs-adapter/pull/799

[OsirisTerje]

@netcorefactory As mentioned in the PR, please raise a separate issue for your PR #799 .