search

swingletree-oss / swingletree

:umbrella: Integrate and observe the results of your CI/CD pipeline tools
https://swingletree-oss.github.io/swingletree/
GNU General Public License v3.0
16 stars 1 forks source link

Enable processing of junit test reports #53

Closed fml2 closed 4 years ago

fml2 commented 4 years ago

It would be cool if there existed a plugin similar to https://github.com/swingletree-oss/plugin-testng that would be able to accept junit test reports and show them in some digested form (total number of tests, details about failed tests). A direct link from within a pool request to the swingletree view would make it even better.

error418 commented 4 years ago

This functionality is handled by plugin-nebula (for gradle-based builds only). At the time this plugin does not write the test results to the check-run details, since it is lacking a report template for this purpose. plugin-nebula reports the amount of total and - if any occurred - skipped and failed tests.

On TestNG

TestNG is compatible with JUnit tests and is shipped with Gradle.

TestNG can automatically recognize and run JUnit tests, so you can use TestNG as a runner for all your existing tests and write new tests using TestNG.

You can enable it by following setting in your build.gradle:

test {
  useTestNG()
}

The resulting report can then be processed with plugin-testng, which attaches a report to your check-run using this template: plugin-testng/templates/report.md.tpl. This template would cover the points you mentioned.

Closing Thoughts

Using TestNG is a good decision to make. TestNG reports are a bit more detailled as their JUnit-counterparts and offers some very interesting features on top of JUnit, which you can look into on the TestNG website.

plugin-testng can provide a solution for TestNG as well as JUnit based tests (which get executed by TestNG). Using one plugin for test-report purposes reduces the maintenance load of swingletree and benefits therefore maintainers as well as swingletree users.

Was this helpful for you?

fml2 commented 4 years ago

I've just tried by including test { useTestNG() } into build.gradle but it did not work. The unit tests were not found. I must have done something wrong.

But even if it did work it would not be ideal since we'd have to adjust all our build scripts (include the useTestNG option since it's not the default).

I'll try again though.

error418 commented 4 years ago

I've just tried by including test { useTestNG() } into build.gradle but it did not work. The unit tests were not found. I must have done something wrong.

Please see the linked "Migrate from JUnit" section of the TestNG docs for further information about this topic

fml2 commented 4 years ago

Would it be feasible to let the testng plugin also process JUnit reports? I.e. recognize the report type by e.g. the root element of the report?

error418 commented 4 years ago

It is planned to release a junit plugin as part of a Sponsorware release. I'll keep you posted inside this issue - stay tuned :blush:

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

error418 commented 4 years ago

This feature is now available via Sponsorware. More information can be found on the docs pages ( https://swingletree-oss.github.io/swingletree/docs/plugins/junit/ )