marklogic-community / marklogic-unit-test

Automated testing for MarkLogic
https://marklogic-community.github.io/marklogic-unit-test/
Other
13 stars 25 forks source link

Update rest endpoint result with codecoverage information and add number of succeeded and failed tests in default output #166

Closed peetkes closed 1 year ago

peetkes commented 1 year ago

The output of the unittest endpoint now only contains the failures for the testcases. It would be convenient to contain also the nr of succeeded and failed tests per testcase. Also when calculatecoverage is enabled it would be nice to have that included as well.

Proposal would be:

<testsuite errors="nr of errors" failures="nr of failures" tests="nr of tests" time="time" timestamp="timestamp" hostname="host" name="suitename">
  <testcase classname="classname.xqy" name="classname.xqy" time="time" failures="nr of failures" tests="number of tests">
    <failure>text of the failure(s)</failure>
    or
    <coverage module="modulename1" wanted="nr of wanted lines" covered="nr of covered lines" missing="nr of missing lines"/>
    <coverage module="modulename2" wanted="nr of wanted lines" covered="nr of covered lines" missing="nr of missing lines"/>
    ..
  </testcase>
</testsuite>

We could filter out all modules that have wanted=0, covered=0 and missing=0

As an alternative we could also add in the linenmbers for covered, wanted and missing and promote those attributes to an element with attribute count. See below

    <coverage module="modulename1">
      <wanted count="nr of wanted lines">lines wanted</wanted>
      <covered count="nr of covered lines">lines covered</covered>
      <missing count="nr of missing lines">lines missing</missing>
  <coverage>
peetkes commented 1 year ago

Will split this one up into two different issues. One for adjusting standrd junit test report reflecting the number of failed and succeeded tests per testcase and one for adding code coverage

rjrudin commented 1 year ago

Closing this, as I don't want to do anything further for now with the code coverage data until it can be made more useful and less misleading. And #176 is now tracking the request for counts of successful assertions.