serenity-bdd / serenity-core

Serenity BDD is a test automation library designed to make writing automated acceptance tests easier, and more fun.
http://serenity-bdd.info
Other
717 stars 514 forks source link

After upgrade serenity to version 4.1.10 I get an error on test scenario details, test report is not generated correctly #3468

Open fericeand opened 3 months ago

fericeand commented 3 months ago

What happened?

After upgrading serenity to 4.1.10 and java to version 21 the serenity report is not generated correctly. When clicking on scenario link from generated report to see the details I'm getting the error "NoSuchKey: The specified key does not exist. status code: 404, request id: 806WFWKCBT5R3W18, host id: YvSr5k8vq1dnmgRlD4G59VIxjg8d2kg6UKwtdHgXnKxoEk9rvk1KTjCU8ZdcfYbeFynHsONCoG0="

I am using Junit5, Serenity 4.1.10, Java 21 and gradle.

What did you expect to happen?

I am expecting to see test result details when clicking on scenario link

Serenity BDD version

4.1.10

JDK version

21

Execution environment

The report is generated through Buildkite and we use Gradle

How to reproduce the bug.

Upgrading serenity to latest version and using junit5

How can we make it happen?

Work on this myself and propose a PR (with Serenity BDD team guidance)

wakaleo commented 3 months ago

Can you provide a sample project that reproduces this issue?

fericeand commented 3 months ago

this is the build.gradle `plugins { id "io.freefair.lombok" version "8.6" id "net.serenity-bdd.serenity-gradle-plugin" version "4.1.10" id "java" id "idea" id 'checkstyle' id 'com.adarshr.test-logger' version '4.0.0' id "org.gradle.test-retry" version "1.5.9" }

group = 'com.fanduel.fgp' version '0.0.1-SNAPSHOT'

defaultTasks 'clean', 'test', 'aggregate'

repositories { mavenCentral() }

ext { slf4jVersion = '2.0.13' serenityCoreVersion = '4.1.10' serenityCucumberVersion = '4.1.10' junitVersion = '5.11.0-M1' junitLauncher = '1.11.0-M1' assertJVersion = '3.25.3' logbackVersion = '1.5.6' jacksonVersion = '2.17.0' jacksonDataFormatVersion = '2.17.0' serenityVersion = '5:4.1.10' aws_version = '2.20.103' }

dependencies { implementation "ch.qos.logback:logback-classic:${logbackVersion}" implementation "com.fasterxml.jackson.datatype:jackson-datatype-jdk8:${jacksonVersion}" implementation "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:${jacksonVersion}" implementation group: 'org.jboss.aerogear', name: 'aerogear-otp-java', version: '1.0.0' implementation "mysql:mysql-connector-java:8.0.32"

testImplementation "net.serenity-bdd:serenity-core:${serenityCoreVersion}"
testImplementation "net.serenity-bdd:serenity-screenplay-rest:${serenityCoreVersion}"
testImplementation "net.serenity-bdd:serenity-ensure:${serenityCoreVersion}"
testImplementation "org.junit.platform:junit-platform-launcher:${junitLauncher}"
testImplementation "org.junit.jupiter:junit-jupiter-engine:${junitVersion}"
testImplementation "org.junit.vintage:junit-vintage-engine:${junitVersion}"
testImplementation "org.assertj:assertj-core:${assertJVersion}"
testImplementation "net.serenity-bdd:serenity-junit${serenityVersion}"
testImplementation "com.fasterxml.jackson.dataformat:jackson-dataformat-xml:${jacksonDataFormatVersion}"

testImplementation platform("software.amazon.awssdk:bom:${aws_version}")
testImplementation 'software.amazon.awssdk:s3'
testImplementation 'software.amazon.awssdk:sts'

}

test { testlogger { theme 'standard-parallel' // task level } testLogging.showStandardStreams = true systemProperties System.getProperties() def test_suite = System.getenv().get("TEST_FLAG") retry { maxRetries = 1 maxFailures = 10 } useJUnitPlatform { excludeTags 'org.junit.jupiter.api.Disabled'

    if ((test_suite != null) && (test_suite != "full_suite")) {
        def tags = test_suite.replace(',', ' & ')
        includeTags tags
    }
}

}

gradle.startParameter.continueOnFailure = true test.finalizedBy(aggregate)

`

fericeand commented 3 months ago

in my case the report is generated but on test scenario details i got the error,, see in the below images

Screenshot 2024-05-20 at 09 01 17 Screenshot 2024-05-20 at 09 01 30 Screenshot 2024-05-20 at 09 01 40
scormaq commented 3 months ago

@fericeand same, after upgrading to 4.1.12, generated report does not contain pages for scenario details. I upgraded from 4.1.0

wakaleo commented 3 months ago

in my case the report is generated but on test scenario details i got the error,, see in the below images Screenshot 2024-05-20 at 09 01 17 Screenshot 2024-05-20 at 09 01 30 Screenshot 2024-05-20 at 09 01 40

"NoSuchKey..." is not a Serenity error. Can you provide a sample project that reproduces this behaviour locally?

wakaleo commented 3 months ago

@fericeand same, after upgrading to 4.1.12, generated report does not contain pages for scenario details. I upgraded from 4.1.0

Try running with the filenames uncompressed - not much has changed between 4.1.12 and 4.1.0 though.

scormaq commented 3 months ago

@wakaleo I played with versions a little bit, it has nothing to do with serenity-core. Issue happens when to upgrade net.serenity-bdd.serenity-gradle-plugin from 4.0.46 to 4.1.3

wakaleo commented 3 months ago

I don't use Gradle so most of the Gradle changes are community driven. Can you have a look at the code and see if you can see anything?

scormaq commented 3 months ago

@wakaleo as I see it's a known issue https://github.com/serenity-bdd/serenity-gradle-plugin/issues/29

fericeand commented 3 months ago

downgraading the serenity gradle plugin version to 4.0.1 fixes the problem

wakaleo commented 3 months ago

downgraading the serenity gradle plugin version to 4.0.1 fixes the problem

Can you tell me the first version where you see the issue occurring?

fericeand commented 3 months ago

I've tried directly with 4.1.10 and it fails. before I've used 3.. version

wakaleo commented 3 months ago

It appears to be a Gradle-specific issue caused by a PR. I will roll back the PR.

wakaleo commented 3 months ago

Can you try with version 4.1.13?

fericeand commented 3 months ago

I get this error with 4.1.13

  | fatal: failed to download artifacts: No artifacts found for downloading

wakaleo commented 3 months ago

Try again in an hour or so - the plugin was deployed recently

fericeand commented 3 months ago

Starting a Gradle Daemon (subsequent builds will be faster)

  |     | FAILURE: Build failed with an exception.   |     | Where:   | Build file '/workdir/build.gradle' line: 79   |     | What went wrong:   | A problem occurred evaluating root project 'fgp-api-tests'.   | > Could not create task ':aggregate'.   | > Could not get unknown property 'generateOutcomes' for extension 'serenity' of type net.serenitybdd.plugins.gradle.SerenityPluginExtension.   |     | * Try:

wakaleo commented 3 months ago

Starting a Gradle Daemon (subsequent builds will be faster)

  |     | FAILURE: Build failed with an exception.   |     | Where:   | Build file '/workdir/build.gradle' line: 79   |     | What went wrong:   | A problem occurred evaluating root project 'fgp-api-tests'.   | > Could not create task ':aggregate'.   | > Could not get unknown property 'generateOutcomes' for extension 'serenity' of type net.serenitybdd.plugins.gradle.SerenityPluginExtension.   |     | * Try:

OK, looks like there's a bug in the PR that tried to fix the bug in the original PR. I am not familiar with Gradle or these changes, so unless someone from the community can test locally and raise a PR, I will just revert to an older version which will have some other known issues.

wakaleo commented 3 months ago

Can you try with 4.1.14?

fericeand commented 3 months ago

same error with no such key for 4.1.14

wakaleo commented 3 months ago

same error with no such key for 4.1.14

It could be project specific, as the reports now work correctly for the test suites I have been testing with (see for example https://github.com/serenity-bdd/screenplay-pattern-todomvc.git). For project-specific support you will need to get some commercial support so we can help with that.

Hammadkhan199 commented 5 days ago

Does serenity version 4.1.14 supports java 21 ? The build passes but not able to run Test Cases

wakaleo commented 5 days ago

Yes

Hammadkhan199 commented 5 days ago

I tried serenity version 4.1.14 with java 18 and java 21. It does not run. But java 11 runs. Any changes needed to be done ?

wakaleo commented 5 days ago

Check your dependencies. Serenity 4.1.x and 4.2.x runs fine with Java 21, but JUnit can be more fiddly.