Cucumber EventListener, Formatter: Adding screenshots

[foursyth]

Summary

I am working with my team to port the HTMLFornatter to the Extent Reporting Framework. A lot, well actually pretty much all the pieces are in place except adding the screenshot functionality.

From what I understand, the EventListener kicks in after all features have been executed. Due to this, the user is unable to attach a screenshot to the relevant test.

Expected Behavior

Like version 1, Cucumber 2+ EventListener or Formatter must allow accessing steps while they are being executes, not afterwards.

Current Behavior

We would like a way to report the step status and add a screenshot if necessary the moment step executes.

Context & Motivation

Due to lack of clarity around the EventListener, a lot of users using custom reporting frameworks like us are unable to use screenshots in our reports.

Your Environment

I use Ubuntu, Win 10.

[foursyth]

Reference to the Formatter port:

https://github.com/extent-framework/extentreports-cucumber4-adapter

[mpkorstanje]

This should help:

https://cucumber.io/blog/2018/09/24/announcing-cucumber-jvm-4-0-0

Because pickles are now executed concurrently TestCaseEvents will no longer arrive in order. To mitigate the impact of this for Eventlistener implementations the Eventbus will record all events and replay them in a canonical order. Implementations of ConcurrentEventListener will receive events as they happen. To help concurrent event listners handle concurrent events all TestCaseEvents now also include the TestCase of which they're part.

Though ideally users use Scenario.write or Scenario.embed rather then talking to extent directly.

[mpkorstanje]

Also beware that these will no longer work:

private ScenarioOutline currentScenarioOutline;
private Examples currentExamples;

When running in parallel there can be multiple current scenarios.

The JUnit and TestNG runners will also assume that they are always executed in parallel even if configured to run serially. There is no way to tell from inside cucumber.

[foursyth]

Hello @mpkorstanje

Thank you for your help!

Can I make a small suggestion? The ConcurrentEventListener should be named as EventListener - because to users EventListener means exactly what the ConcurrentEventListener is doing.

And, EventListener is basically running after the Feature or Suite, so it can be called as SuiteListener as it is not running as per the events, but as per aggregations. It will avoid a lot of confusion. Would love to hear your feedback too. Thanks again.

[mpkorstanje]

Cheers. Considered that but the Event Listener existed first. Parallel support, and the requirement to support concurrent test case events came after.

In theory the source and time of events shouldn't make a difference in an event based system though. Because of this most existing plugins didn't need to be upgraded.

Also Cucumber emits Embed and Write Events when information is embedded or written in to a scenario. So I can't quite work out why you you'd need a concurrent event listener in the first place.

[foursyth]

Thank you!

One last question, hopefully this is a quick one for you.

How is the EmbedEvent being called in the reporters? What needs to happen on the feature or in the stepdefs to invoke this? Can we still using the @aafter annotation to embed screenshots manually?

[foursyth]

Actually. I will open a new ticket as this is unrelated.

[deena-p]

@mpkorstanje Even I have the same question, how to attach screenshot to the steps using the EmbedEvent?

[foursyth]

Hello @deena-p please see the response here:

https://github.com/cucumber/cucumber-jvm/issues/1504#issuecomment-440231377

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.