Xray-App / xray-junit-extensions

Enhanced JUnit 5 integration with Xray Test Management for Jira
Eclipse Public License 2.0
16 stars 8 forks source link

Xray Plugin: when we use Xray plugin in Serenity framework, Xray listener suppresses any exception that test shows and marking the build successful although there are exceptions in the test. #27

Closed InduKrish closed 1 year ago

InduKrish commented 1 year ago

when we use Xray plugin in Serenity framework, Xray listener suppresses any exception that test shows and marking the build successful.

We observed that the xary plugin suppresses any exception that the test throws and producing inconsistent build status when we run the tests using Serenity -junit 5 framework and using Xray-junit5 plugin to produce an xml for xray reporting..

although there were exceptions in the code in Serenity -junit 5 framework, the test build was marked successful, and after removing the xray listener, we noticed that the serenity-junit 5 marked the build failed whenever there are exception in the build as expected.

Can you please advise why Xray listener is causing the build status to be not appropriate?, when used in Serenity framework.

Listener

org.junit.platform.launcher.TestExecutionListener app.getxray.xray.junit.customjunitxml.EnhancedLegacyXmlReportGeneratingListener

build.gradle Serenity-junit5

testImplementation 'net.serenity-bdd:serenity-core' testImplementation 'net.serenity-bdd:serenity-junit5' testImplementation 'net.serenity-bdd:serenity-rest-assured' testImplementation 'org.junit.jupiter:junit-jupiter-api' testImplementation 'org.junit.platform:junit-platform-suite-engine' testImplementation 'app.getxray:xray-junit-extensions'

bitcoder commented 1 year ago

I'm not a frequent Serenity user so to be able to help I need help from someone else most probably. Nevertheless, it would help if you could create a full working example, for example on GitHb repo, that could show this behavior and thus be able to replicate it as the information you provided is very scarce.

bitcoder commented 1 year ago

If no additional feedback is provided in the upcoming days, I'll close this ticket

bitcoder commented 1 year ago

Closing this for lack of feedback