search

Xray-App / xray-testng-extensions

Enhanced TestNG integration with Xray Test Management for Jira
MIT License
9 stars 5 forks source link

configuration for including only annotated tests on the report #21

Closed bitcoder closed 3 months ago

bitcoder commented 4 months ago

Include on the generated report only tests annotated with @XrayTest or @Requirement, by providing a configuration option report_only_annotated_tests (default would be "false").

bitcoder commented 4 months ago

Analysis:

  1. TestNG XML output, using the XrayListener
    • the current implementation uses the XMLReporter provided by TestNG itself, and then uses a custom XrayListener to process the XrayTest and Requirement annotations. Complementary, and optionally, we can also use the XrayReportListener to embed attachments (e.g., screenshots, logs) on the test results.
    • TestNG's XMLReporter uses several classes, like the XMLSuiteResultWriter to write the results to the typical testng-results.xml report file. There's no way to exclude certain tests from appearing on the generated report. We al
    • possible way forward: replicate the code from upstream project and reimplement the XMLReporter to address this project needs; we need to check how viable this would be (maintenance & license).
  2. Xray JSON output, using the custom XrayJsonReporter class.
    • since this class is fully custom, and provided by this project, we can adjust its behaviour.
    • possible way forward: adjust the generateReport() method to look for @XrayTest or @Requirement annotations and decide wether to include the tests or not on the generated Xray JSON report