Frameworkium / frameworkium-core

Framework for writing maintainable Selenium and REST API tests in Java.
https://frameworkium.github.io/
Apache License 2.0
160 stars 82 forks source link

github413: Update listeners for Allure reporting #414

Closed emerson-b closed 2 years ago

emerson-b commented 2 years ago

Fixes #413

The latest versions of TestNG have changed the order that listeners are triggered by inverting the array of listeners on a test finish event. This means that Allure reports were stopping for a test before attachments could be added.

By removed the listeners from the Base tests and moving them to META-INF/services/org.testng.ITestNGListener - testNG will instantiate the listeners using service loading. This methods forces the listeners, both 1st and 3rd party, to be picked up and used in the order given in the file. By setting allure to be the first listener, when the test ends it is the last thing to close and attachments now work again.

codecov[bot] commented 2 years ago

Codecov Report

Merging #414 (af3704a) into master (7bf7219) will not change coverage. The diff coverage is 50.00%.

@@            Coverage Diff            @@
##             master     #414   +/-   ##
=========================================
  Coverage     49.50%   49.50%           
  Complexity      477      477           
=========================================
  Files           114      114           
  Lines          2818     2818           
  Branches        168      168           
=========================================
  Hits           1395     1395           
  Misses         1339     1339           
  Partials         84       84           
Impacted Files Coverage Δ
...a/com/frameworkium/core/api/tests/BaseAPITest.java 100.00% <ø> (ø)
...meworkium/core/htmlelements/element/FileInput.java 0.00% <0.00%> (ø)
...ava/com/frameworkium/core/ui/tests/BaseUITest.java 52.63% <ø> (ø)
...eworkium/core/ui/listeners/ScreenshotListener.java 45.45% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 789e16c...af3704a. Read the comment docs.

Mjl33 commented 2 years ago

Thanks for the pull request. There seems to be a big drop in code coverage for the amount of code change. I haven't had a chance to take a closer look, but I suspect class static removal has something to do with it

If you could please see if you can improve code coverage, else I am happy for this to merge.

emerson-b commented 2 years ago

@Mjl33 I'm going to refactor out the static changes as that's a separate issue, so we can see if it is the static declaration causing the coverage changes. I'll drop those in its own commit after