🐛 Bug: xUnit XML format is actually JUnit's schema

[andyleejordan]

Prerequisites

• [x] Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
• [x] Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
• [x] 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
• [x] Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend that you not install Mocha globally.

Description

Hi there,

I feel like I'm crazy, but as far as I can tell, the XML output for Mocha's xUnit reporter is actually JUnit's schema. In short, it's emitting something like:

<testsuite ... >
  <testcase ... />
<testsuite />

Which is most definitely the documented JUnit XML schema. Mocha's own tests for the xUnit output confirm this what it's intended to emit:

https://github.com/mochajs/mocha/blob/0ea732c1169c678ef116c3eb452cc94758fff150/test/reporters/xunit.spec.js#L322-L453

However, the two versions of xUnit XML schema (v1 and v2) look radically different from this! For one, the top-level element must be <assemblies>, which doesn't exist in Mocha's codebase. And neither use <testcase> nor <testsuite> (they both use a <test> element with further information provided by subelements).

I don't know how I'm the first to notice this when this project is used by 1.4 million other things on GitHub, which is why I think I must be wrong, despite staring at the documentation and tests I've just linked that justify this bug is real.

[andyleejordan]

Hey @juergba, any follow up on this, am I crazy?

[miensol]

I'm relieved. I thought I'm missing something as well. Naming things is indeed hard.

[miensol]

What's even more interesting is that apparently this is not the only case where the names about xunit and junit aren't correct. I was trying to verify if Reportportal supports xunit XML files import. I found this class junit/XunitImportHandler.java which only supports junit XML format...

[EnricoMi]

Yes, this is JUnit XML, not XUnit XML.

[EnricoMi]

@jkrall @tj @boneskull what do you think?

[JoshuaKGoldberg]

cc @mochajs/maintenance-crew - looks like this is a long-standing issue that has tripped quite a few folks up. My intuition is we'd want to make a breaking change fix in the next major version that:

• Renames the xunit reporter to junit, since it's emitting jUnit XML
• Creates a new xunit reporter that actually emits xUnit XML

Thoughts?

[Maverick099]

Do we have a timeline for when this issue will be closed?