search

AOEpeople / geb-spock-reports

Integrates Geb screenshots into the spock-reports library
Apache License 2.0
36 stars 11 forks source link

.html spec reports not generated with spock-reports 2.0-RC2 #28

Closed treyturner closed 4 years ago

treyturner commented 4 years ago

Hello! I was super stoked that my request for Spock 2.0 support from spock-reports was addressed and built into an RC, and that seems to be working great! Unfortunately it also appears that my UI test project which additionally implements geb-spock-reports doesn't seem to play nice with it. Though I get a main index summarizing results of my UI tests, clicking on details for any specification throws a 404 because the linked .html is not generated. By contrast, the AsciiDoc IReportCreator I've implemented seems to be working, so I think the problem is limited to geb-spock-reports.

I'd love to provide additional detail about the failure, but I'm afraid none is apparent - any tips on where I could look or how to enable additional logging for the geb-spock-reports IReportCreator?

tilmanginzel commented 4 years ago

Hi @treyturner, thanks for the issue! I was watching your spock-reports issue and wanted to check if geb-spock-reports also needs to be updated accordingly. Logging is basically non-existant as of now, the template report creation will fail silently.

I suppose that there have been breaking changes in Spock 2.0, so I might need to incorporate those in my template. I should have some time for this the coming weekend. :)

tilmanginzel commented 4 years ago

Hey @treyturner, seems like the issue got auto-closed by merging my changes. I have published version 0.3.0-RC1, please test if everything works as expected now and if not, just reopen the issue. :)

treyturner commented 4 years ago

@tilmanginzel It doesn't seem that I'm able to reopen the issue. I can create a new issue if desired, but here's what I'm seeing.

I do get .html spec report generation now, but the artifacts show up together as a group at the bottom of the report (in a group called Unassigned Geb Artifacts) instead of attaching to their respective features. I also notice it's attempting to link three screenshots when only one feature failed. All artifacts have the same name: fixture-failure. The links to the second and third screenshots are broken, because the artifacts don't exist - presumably because the features passed.

image

ssulei7 commented 4 years ago

@tilmanginzel It doesn't seem that I'm able to reopen the issue. I can create a new issue if desired, but here's what I'm seeing.

I do get .html spec report generation now, but the artifacts show up together as a group at the bottom of the report (in a group called Unassigned Geb Artifacts) instead of attaching to their respective features. I also notice it's attempting to link three screenshots when only one feature failed. All artifacts have the same name: fixture-failure. The links to the second and third screenshots are broken, because the artifacts don't exist - presumably because the features passed.

image

I'm facing the same issue as well.

tilmanginzel commented 4 years ago

Hi @treyturner, @ssulei7, thanks for the info. I can reproduce it with my sample project. No idea how I have missed that. :/

I was able to pin it down to an invalid label name in my generated geb-report-info.json. Previously, it was the name of the feature, e.g. simple screenshot test, now it's just always fixture, so my template cannot find the respective artifacts by that name. Will fix it as soon as possible.

tilmanginzel commented 4 years ago

Intermediate result: The test name is being provided by GebReportingSpec#L26 and then used to create a report label.

On previous versions, this value was set with the name of the feature, e.g.:

Bildschirmfoto 2020-05-27 um 19 48 25

With the latest version, this value is always null, resulting in the default value fixture:

Bildschirmfoto 2020-05-27 um 19 50 31

Nothing has changed on the side of Geb in the last years, and I need to investigate what this @Rule annotation does. There seems to be some upstream issue, that this value is not set properly. Might be related to using a dependency mix of JUnit 4 and JUnit 5.

tilmanginzel commented 4 years ago

@Rule TestName does not work in JUnit 5 anymore, and it seems like Geb does not yet support JUnit 5: https://github.com/geb/issues/issues/539

So I fear we are blocked here for now. :/ Maybe this library could provide it's own GebSpockReportingSpec, but not sure if that would be the right approach.

tilmanginzel commented 4 years ago

Hey @treyturner, @ssulei7, I have found an easy workaround. Spock provides a spock-junit4 compatibility module. Simply adding the dependency provides @Rule TestName annotation support! Can you check if adding it to your build.gradle fixes the issue? I would then add some documentation to the README.

dependencies {
  testCompile "org.spockframework:spock-core:2.0-M2-groovy-3.0"
  testCompile "org.spockframework:spock-junit4:2.0-M2-groovy-3.0"
  ...
}
ssulei7 commented 4 years ago

Hey @treyturner, @ssulei7, I have found an easy workaround. Spock provides a spock-junit4 compatibility module. Simply adding the dependency provides @Rule TestName annotation support! Can you check if adding it to your build.gradle fixes the issue? I would then add some documentation to the README.

dependencies {
  testCompile "org.spockframework:spock-core:2.0-M2-groovy-3.0"
  testCompile "org.spockframework:spock-junit4:2.0-M2-groovy-3.0"
  ...
}

That ended up working for me!

tilmanginzel commented 4 years ago

Thanks for checking! I have updated the README and will close this issue again. If you encounter anything else, feel free to reopen or create a new one. :)