search

tcunit / TcUnit

An unit testing framework for Beckhoff's TwinCAT 3
Other
258 stars 72 forks source link

feat(xUnitXmlPublisher): Include assertion message in test output #208

Open stefanbesler opened 1 year ago

stefanbesler commented 1 year ago

This PR write the actual assertion failure to the output of xUnitXmlPublisher. Previously only the message from test was written into message.

@dfreiberger fyi

stefanbesler commented 9 months ago

Maybe some explanation on how to verify this:

image

If any test fails, the C:\tcunit_xunit_testresults.xml will now correctly show the EXP and ACT values in the assertion message

<testsuites disabled="" failures="116" tests="144" time="6.1428994">
   <testsuite id="0" name="PRG_TEST.PrimitiveTypes" tests="46" failures="23" time="1.4893e-3">
      <testcase name="Test_ANY_Equals" classname="PRG_TEST.PrimitiveTypes" time="2.35e-5" status="PASS" />
      <testcase name="Test_ANY_Differ_DataType" classname="PRG_TEST.PrimitiveTypes" time="1.47e-5" status="FAIL">
         <failure message="Values differ; EXP: (Type class = INT), ACT: (Type class = UINT)" type="Array2D_LREAL" />
      </testcase>
      <testcase name="Test_BOOL_Equals" classname="PRG_TEST.PrimitiveTypes" time="3.8e-6" status="PASS" />
      <testcase name="Test_BOOL_Differ" classname="PRG_TEST.PrimitiveTypes" time="1.18e-5" status="FAIL">
         <failure message="Values differ; EXP: TRUE, ACT: FALSE" type="Array2D_LREAL" />
      </testcase>
      <testcase name="Test_BYTE_Equals" classname="PRG_TEST.PrimitiveTypes" time="4.4e-6" status="PASS" />
      <testcase name="Test_BYTE_Differ" classname="PRG_TEST.PrimitiveTypes" time="1.14e-5" status="FAIL">
         <failure message="Values differ; EXP: 0xAB, ACT: 0xCD" type="Array2D_LREAL" />
      </testcase>

Without the PR, failure > message only shows "Values differ"

sagatowski commented 9 months ago

I'll start to look into this as soon as your previous PR is fixed/completed.

See: https://github.com/tcunit/TcUnit/issues/210 & https://github.com/tcunit/TcUnit/issues/226

sagatowski commented 9 months ago

@stefanbesler Have you noticed the problems described in #226?

dfreiberger commented 9 months ago

@stefanbesler thanks for this, it will be good to have these more granular results.

TimSalzmann commented 9 months ago

@stefanbesler, Hi, I really want to have this feature. If I can help, please let me know

sagatowski commented 9 months ago

I agree, this is a great feature. I do however have to get the stuff fixed that will be included in version 1.3 first. For this milestone (https://github.com/tcunit/TcUnit/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+1.3.0.0%22) there is only one ticket left to be solved, and this one has to be solved first before I look into integrating more PRs.

@dfreiberger & @TimSalzmann, you are of course welcome to help me figure out the problem!

sagatowski commented 9 months ago

Realized that the discussion in this PR went very OffTopic, cleared it up. Sorry for that!

@stefanbesler I'm working on the last pieces to release 1.3. After that, I will look into this PR.