Optionally emit JUnit testcase (failure) messages to system-err

[gfoidl]

Fixes https://github.com/gfoidl/trx2junit/issues/97

When CLI-flag --junit-messages-to-system-err is set or when WorkerOptions.JUnitMessagesToSystemErr is set, then failure messages are emitted to system-err on that testcase too.

E.g. when the flag is not set, then the junit-file will look like:

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite name="NUnitSample.SimpleTests" hostname="xyz" package="not available" id="0" tests="1" failures="1" errors="0" skipped="0" time="1.074" timestamp="2019-04-19T16:06:41">
        <properties />
        <testcase name="Failing_test" classname="NUnitSample.SimpleTests" time="0.059" status="0">
            <failure message="Failing for demo purposes" type="not specified">
                at NUnitSample.SimpleTests.Failing_test() in ...trx2junit\samples\NUnitSample\SimpleTests.cs:line 21
            </failure>
        </testcase>
        <system-out />
        <system-err />
    </testsuite>
</testsuites>

With --junit-messages-to-system-err set the junit-file looks like:

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite name="NUnitSample.SimpleTests" hostname="xyz" package="not available" id="0" tests="1" failures="1" errors="0" skipped="0" time="1.074" timestamp="2019-04-19T16:06:41">
        <properties />
        <testcase name="Failing_test" classname="NUnitSample.SimpleTests" time="0.059" status="0">
            <failure message="Failing for demo purposes" type="not specified">
                at NUnitSample.SimpleTests.Failing_test() in ...trx2junit\samples\NUnitSample\SimpleTests.cs:line 21
            </failure>
            <system-err>
                Failing for demo purposes
            </system-err>
        </testcase>
        <system-out />
        <system-err>Failing for demo purposes</system-err>
    </testsuite>
</testsuites>

[gfoidl]

@juwens here's my stab on it.

Please try it with the instructions below, and tell me if it fits your needs. Then I'd like to create a real release of it.

For the package see trx2junit.Core v2.1.0-preview-1 The feed is https://pkgs.dev.azure.com/gh-gfoidl/github-Projects/_packaging/gfoidl-public/nuget/v3/index.json (more info in the link above).

For the tool:

# deinstall current version
dotnet tool uninstall -g trx2junit

# install preview
dotnet tool install -g trx2junit --version 2.1.0-preview-1 --add-source https://pkgs.dev.azure.com/gh-gfoidl/github-Projects/_packaging/gfoidl-public/nuget/v3/index.json

---

In https://github.com/gfoidl/trx2junit/issues/97 we talked about system-out, but while implementing it system-err made more sense to me (due the err). If gitlab doesn't like it and only want to read from system-out then I'll update accordingly.

[juwens]

Hey @gfoidl it's hard to say what is correct in this case.

Your change wouldn't fit MY needs. But maybe i'm interpreting something wrong.

Gitlab in it's Test-Result UI, when clicking on view details of a failed test, gitlab seems to only display the inner-content of the failure node. Though it says "standard output", which is highly confusing.

So what I'm currently doing is: rewrite the junit xml and put everything in the failure node.

Essentially:   

failureNode.InnerText = 
    failureNode.Attributes["message"].Value + "\n" 
   + failureNode.InnerText 
   + "\n\nStandard Out: \n"
   + systemOutNode.InnerText
   + "\n\nStandard Err: \n"
   + systemErrNode.InnerText;

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite name="NUnitSample.SimpleTests" hostname="xyz" package="not available" id="0" tests="1" failures="1" errors="0" skipped="0" time="1.074" timestamp="2019-04-19T16:06:41">
        <properties />
        <testcase name="Failing_test" classname="NUnitSample.SimpleTests" time="0.059" status="0">
            <failure message="Failing for demo purposes" type="not specified">
                Failing for demo purposes
                at NUnitSample.SimpleTests.Failing_test() in ...trx2junit\samples\NUnitSample\SimpleTests.cs:line 21

                Standard Out: 
                text from stdout

                Standard Error:
                text from stderr
            </failure>    
            <system-out>text from stdout</system-out>
            <system-err>text from stderr</system-err>
        </testcase>
    </testsuite>
</testsuites>

[juwens]

Actual code (i don't care for system-err at the moment):
 

int Main()
{
    foreach (var file in files)
    {
        XmlDocument doc = new();
        doc.Load(file.FullPath);
        var failureNodes = doc.SelectNodes("/testsuites/testsuite/testcase/failure");
        foreach (XmlNode failureNode in failureNodes)
        {
            PrependMessageAttributeContentToFailureNode(failureNode);
            AppendSystemOutNodeContentToFailureNode(failureNode);
        }
        doc.Save(file.FullPath);
    }
}

void AppendSystemOutNodeContentToFailureNode(XmlNode failureNode)
{
    var testcaseNode = failureNode.ParentNode;
    var systemOutNode = testcaseNode.SelectSingleNode("system-out");
    failureNode.InnerText += $"\n\nStandard Output (stdout):\n\n{systemOutNode.InnerText}";
}

[gfoidl]

I think no I got the problem by

failureNode.InnerText = 
    failureNode.Attributes["message"].Value + "\n" 
   + failureNode.InnerText 
   + "\n\nStandard Out: \n"
   + systemOutNode.InnerText
   + "\n\nStandard Err: \n"
   + systemErrNode.InnerText;

Will provide a updated preview soon (but not today).

[gfoidl]

I'm going to close this PR, as

• it doesn't fix the underlying problem
• it's a bit awkward to have the fail-reason in system-err -- it's not what it's meant for

PR https://github.com/gfoidl/trx2junit/pull/101 provides a better solution.