search

Randy424 / console

The user interface for the cluster management portion of open-cluster-management.
Apache License 2.0
0 stars 0 forks source link

Define required test spec format for test reporting #12

Open dtthuynh opened 1 year ago

Randy424 commented 1 year ago

Test result reporting - Do developers need specific formatting/information... See Vincent for his use of JUnit (XML)

Randy424 commented 1 year ago

Update: not started :)

dtthuynh commented 1 year ago

sample of an existing result from Jenkins:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Mocha Tests" time="2277.7990" tests="2" failures="1">
  <testsuite name="Root Suite" timestamp="2023-04-26T21:35:24" tests="0" file="cypress/tests/postUpgrade/postUpgrade.spec.js" time="0.0000" failures="0">
  </testsuite>
  <testsuite name="post upgrade validation" timestamp="2023-04-26T21:35:24" tests="2" time="2277.6440" failures="1">
    <testcase name="RHACM4K-857: CLC: Post-Upgrade - Verify all managed clusters from before upgrade" time="71.4650" classname="post upgrade validation RHACM4K-857: CLC: Post-Upgrade - Verify all managed clusters from before upgrade">
    </testcase>
    <testcase name="RHACM4K-862: CLC: Post Upgrade - create and import new clusters after upgrade" time="0.0000" classname="post upgrade validation RHACM4K-862: CLC: Post Upgrade - create and import new clusters after upgrade">
      <failure message="Timed out retrying after 10000ms: expected &apos;[ &lt;button.pf-c-table__button&gt;, 1 more... ]&apos; to contain &apos;Ready&apos;" type="AssertionError"><![CDATA[AssertionError: Timed out retrying after 10000ms: expected '[ <button.pf-c-table__button>, 1 more... ]' to contain 'Ready'
    at Object.checkCluster (https://multicloud-console.apps.acmqe-258fc2-zupgrade.gcp.dev09.red-chesterfield.com/__cypress/tests?p=cypress/tests/postUpgrade/postUpgrade.spec.js:5544:85)
    at Context.eval (https://multicloud-console.apps.acmqe-258fc2-zupgrade.gcp.dev09.red-chesterfield.com/__cypress/tests?p=cypress/tests/postUpgrade/postUpgrade.spec.js:3767:58)]]></failure>
    </testcase>
  </testsuite>
</testsuites>
dtthuynh commented 1 year ago

this allows our reportPortal and Polarion to be updated with the testcase result that's ran:

image
Randy424 commented 1 year ago

Should not be difficult to install and test. This enables integration with Polarion through XML artifact of test output.