search

DevExpress / testcafe-reporter-xunit

This is the xUnit reporter plugin for TestCafe.
https://testcafe.io
MIT License
9 stars 30 forks source link

Illegal characters are not stripped before writing XML #33

Closed malloryavvir closed 1 year ago

malloryavvir commented 1 year ago

Per the XML 1.1 spec, XML documents may contain all characters in Char except those in RestrictedChar (whether encoded as a raw character or as numeric character references):

Char ::= [#x1-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]
RestrictedChar ::= [#x1-#x8] | [#xB-#xC] | [#xE-#x1F] | [#x7F-#x84] | [#x86-#x9F]

Here is a sample from an XML file that CircleCI could not parse. Notice that  falls within [#xE-#x1F] and thus prevents the document from being well-formed.

Here are the accessible roles:

  button:

  Name "REDACTED":
  [36m<div[39m
AlexKamaev commented 1 year ago

I did not manage to reproduce the issue since I do not have access to CircleCI. Please create a sample witch demonstrate the issue or share a detailed instruction on how we can reproduce it. Please also note that you can contribute in this repository, so if you want to fix the issue yourself you can create a PR. Also, you can use a workaround for the described behavior. Please refer to the following help topic: https://testcafe.io/documentation/404388/guides/advanced-guides/modify-reporter-output I prepared a sample which demonstrates how to replace test name with custom text in the reporter to demonstrate this approach. .testcaferc.js:

function hookFunction (writeInfo) {
    if (writeInfo.initiator === 'reportTestDone') {
        writeInfo.formattedText = writeInfo.formattedText.replace(/.*/, 'My Custom Test Name');
    }
}

module.exports = {
    hooks: {
        reporter: {
            onBeforeWrite: {
                'spec': hookFunction,
            },
        },
    },
};
malloryavvir commented 1 year ago

I tried to make a failing test yesterday but the existing tests were not passing for me. Am I running them wrong, or are the tests really failing right now?

I did not manage to reproduce the issue since I do not have access to CircleCI.

For the record you don't need to use CircleCI, you can paste some XML containing  into any XML parser. xmlstarlet validate output.xml will produce an error, for example.

AlexKamaev commented 1 year ago

I tried to make a failing test yesterday but the existing tests were not passing for me. Am I running them wrong, or are the tests really failing right now?

Sorry, I did not understand what tests you mean.

For the record you don't need to use CircleCI, you can paste some XML containing  into any XML parser. xmlstarlet validate output.xml will produce an error, for example.

Please share with us a test file that causes the described behavior.

I need to mention that at present we already planned our workload for the near future, so we cannot give any estimates on when we start working on the issue. However, we can accept a PR from you, if you make it.

malloryavvir commented 1 year ago

Sorry, I did not understand what tests you mean.

The below is the output I get using yarn test or npm test using Node v16.14.0. The tests also fail with Node 18 and 20.

yarn run v1.22.19
$ npm run lint && mocha

> testcafe-reporter-xunit@2.2.2 lint
> eslint .

  1) Should produce report with colors
  2) Should produce report without colors

  0 passing (49ms)
  2 failing

  1) Should produce report with colors:

      AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected ... Lines skipped

  '<?xml version="1.0" encoding="UTF-8" ?>\n' +
    '<testsuite name="TestCafe Tests: Chrome 41.0.2227 / Mac OS X 10.10.1, Firefox 47 / Mac OS X 10.10.1" tests="6" failures="2" skipped="1" errors="2" time="925" timestamp="Thu, 01 Jan 1970 00:15:25 GMT" >\n' +
...
    '            at Object.<anonymous> (some-file:1:1)\n' +
    '            at async formattedImport (some-file:1:1)\n' +
+   '            at async Object.exports.requireOrImport (some-file:1:1)\n' +
+   '            at async Object.exports.loadFilesAsync (some-file:1:1)\n' +
-   '            at async exports.requireOrImport (some-file:1:1)\n' +
-   '            at async exports.loadFilesAsync (some-file:1:1)\n' +
    '            at async singleRun (some-file:1:1)\n' +
+   '            at async Object.exports.handler (some-file:1:1)\n' +
-   '            at async exports.handler (some-file:1:1)\n' +
    '\n' +
    '      2) The specified selector does not match any element in the DOM tree.\n' +
...
    '            at Object.<anonymous> (some-file:1:1)\n' +
    '            at async formattedImport (some-file:1:1)\n' +
+   '            at async Object.exports.requireOrImport (some-file:1:1)\n' +
+   '            at async Object.exports.loadFilesAsync (some-file:1:1)\n' +
-   '            at async exports.requireOrImport (some-file:1:1)\n' +
-   '            at async exports.loadFilesAsync (some-file:1:1)\n' +
    '            at async singleRun (some-file:1:1)\n' +
+   '            at async Object.exports.handler (some-file:1:1)\n' +
-   '            at async exports.handler (some-file:1:1)\n' +
    '    ]]>\n' +
    '    </failure>\n' +
...
    '            at Object.<anonymous> (some-file:1:1)\n' +
    '            at async formattedImport (some-file:1:1)\n' +
+   '            at async Object.exports.requireOrImport (some-file:1:1)\n' +
+   '            at async Object.exports.loadFilesAsync (some-file:1:1)\n' +
-   '            at async exports.requireOrImport (some-file:1:1)\n' +
-   '            at async exports.loadFilesAsync (some-file:1:1)\n' +
    '            at async singleRun (some-file:1:1)\n' +
+   '            at async Object.exports.handler (some-file:1:1)\n' +
-   '            at async exports.handler (some-file:1:1)\n' +
    '    ]]>\n' +
...
    '  ]]>\n' +
    '  </system-out>\n' +
    '</testsuite>'
      + expected - actual

                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at async formattedImport (some-file:1:1)
      -            at async Object.exports.requireOrImport (some-file:1:1)
      -            at async Object.exports.loadFilesAsync (some-file:1:1)
      +            at async exports.requireOrImport (some-file:1:1)
      +            at async exports.loadFilesAsync (some-file:1:1)
                   at async singleRun (some-file:1:1)
      -            at async Object.exports.handler (some-file:1:1)
      +            at async exports.handler (some-file:1:1)

             2) The specified selector does not match any element in the DOM tree.

                   | Selector
--
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at async formattedImport (some-file:1:1)
      -            at async Object.exports.requireOrImport (some-file:1:1)
      -            at async Object.exports.loadFilesAsync (some-file:1:1)
      +            at async exports.requireOrImport (some-file:1:1)
      +            at async exports.loadFilesAsync (some-file:1:1)
                   at async singleRun (some-file:1:1)
      -            at async Object.exports.handler (some-file:1:1)
      +            at async exports.handler (some-file:1:1)
           ]]>
           </failure>
         </testcase>
         <testcase classname="First fixture" file="./fixture1.js" name="Third test in first fixture" time="74">
--
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at async formattedImport (some-file:1:1)
      -            at async Object.exports.requireOrImport (some-file:1:1)
      -            at async Object.exports.loadFilesAsync (some-file:1:1)
      +            at async exports.requireOrImport (some-file:1:1)
      +            at async exports.loadFilesAsync (some-file:1:1)
                   at async singleRun (some-file:1:1)
      -            at async Object.exports.handler (some-file:1:1)
      +            at async exports.handler (some-file:1:1)
           ]]>
           </failure>
         </testcase>
         <system-out>

      at Context.<anonymous> (test/test.js:13:12)
      at processImmediate (node:internal/timers:466:21)
      at process.topLevelDomainCallback (node:domain:152:15)
      at process.callbackTrampoline (node:internal/async_hooks:128:24)

  2) Should produce report without colors:

      AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected ... Lines skipped

  '<?xml version="1.0" encoding="UTF-8" ?>\n' +
    '<testsuite name="TestCafe Tests: Chrome 41.0.2227 / Mac OS X 10.10.1, Firefox 47 / Mac OS X 10.10.1" tests="6" failures="2" skipped="1" errors="2" time="925" timestamp="Thu, 01 Jan 1970 00:15:25 GMT" >\n' +
...
    '            at Object.<anonymous> (some-file:1:1)\n' +
    '            at async formattedImport (some-file:1:1)\n' +
+   '            at async Object.exports.requireOrImport (some-file:1:1)\n' +
+   '            at async Object.exports.loadFilesAsync (some-file:1:1)\n' +
-   '            at async exports.requireOrImport (some-file:1:1)\n' +
-   '            at async exports.loadFilesAsync (some-file:1:1)\n' +
    '            at async singleRun (some-file:1:1)\n' +
+   '            at async Object.exports.handler (some-file:1:1)\n' +
-   '            at async exports.handler (some-file:1:1)\n' +
    '\n' +
    '      2) The specified selector does not match any element in the DOM tree.\n' +
...
    '            at Object.<anonymous> (some-file:1:1)\n' +
    '            at async formattedImport (some-file:1:1)\n' +
+   '            at async Object.exports.requireOrImport (some-file:1:1)\n' +
+   '            at async Object.exports.loadFilesAsync (some-file:1:1)\n' +
-   '            at async exports.requireOrImport (some-file:1:1)\n' +
-   '            at async exports.loadFilesAsync (some-file:1:1)\n' +
    '            at async singleRun (some-file:1:1)\n' +
+   '            at async Object.exports.handler (some-file:1:1)\n' +
-   '            at async exports.handler (some-file:1:1)\n' +
    '    ]]>\n' +
    '    </failure>\n' +
...
    '            at Object.<anonymous> (some-file:1:1)\n' +
    '            at async formattedImport (some-file:1:1)\n' +
+   '            at async Object.exports.requireOrImport (some-file:1:1)\n' +
+   '            at async Object.exports.loadFilesAsync (some-file:1:1)\n' +
-   '            at async exports.requireOrImport (some-file:1:1)\n' +
-   '            at async exports.loadFilesAsync (some-file:1:1)\n' +
    '            at async singleRun (some-file:1:1)\n' +
+   '            at async Object.exports.handler (some-file:1:1)\n' +
-   '            at async exports.handler (some-file:1:1)\n' +
    '    ]]>\n' +
...
    '  ]]>\n' +
    '  </system-out>\n' +
    '</testsuite>'
      + expected - actual

                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at async formattedImport (some-file:1:1)
      -            at async Object.exports.requireOrImport (some-file:1:1)
      -            at async Object.exports.loadFilesAsync (some-file:1:1)
      +            at async exports.requireOrImport (some-file:1:1)
      +            at async exports.loadFilesAsync (some-file:1:1)
                   at async singleRun (some-file:1:1)
      -            at async Object.exports.handler (some-file:1:1)
      +            at async exports.handler (some-file:1:1)

             2) The specified selector does not match any element in the DOM tree.

                   | Selector
--
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at async formattedImport (some-file:1:1)
      -            at async Object.exports.requireOrImport (some-file:1:1)
      -            at async Object.exports.loadFilesAsync (some-file:1:1)
      +            at async exports.requireOrImport (some-file:1:1)
      +            at async exports.loadFilesAsync (some-file:1:1)
                   at async singleRun (some-file:1:1)
      -            at async Object.exports.handler (some-file:1:1)
      +            at async exports.handler (some-file:1:1)
           ]]>
           </failure>
         </testcase>
         <testcase classname="First fixture" file="./fixture1.js" name="Third test in first fixture" time="74">
--
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at Object.<anonymous> (some-file:1:1)
                   at async formattedImport (some-file:1:1)
      -            at async Object.exports.requireOrImport (some-file:1:1)
      -            at async Object.exports.loadFilesAsync (some-file:1:1)
      +            at async exports.requireOrImport (some-file:1:1)
      +            at async exports.loadFilesAsync (some-file:1:1)
                   at async singleRun (some-file:1:1)
      -            at async Object.exports.handler (some-file:1:1)
      +            at async exports.handler (some-file:1:1)
           ]]>
           </failure>
         </testcase>
         <system-out>

      at Context.<anonymous> (test/test.js:23:12)
      at processImmediate (node:internal/timers:466:21)
      at process.topLevelDomainCallback (node:domain:152:15)
      at process.callbackTrampoline (node:internal/async_hooks:128:24)

info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
AlexKamaev commented 1 year ago

Please share test code, which we can run on our machines and get the invalid XML file.