search

prashant-ramcharan / courgette-jvm

Multiprocess | Parallel Cucumber-JVM | Parallelize your Java Cucumber tests on a feature level or on a scenario level.
MIT License
134 stars 39 forks source link

How to have the status as WARNING in the Courgette Report? #386

Closed bhala-ki closed 1 year ago

bhala-ki commented 1 year ago

I would like to have the status as WARNING instead of PASSED OR FAILED OR FAILED AFTER RERUN statuses. Is this something possible? If yes, please guide me.

prashant-ramcharan commented 1 year ago

What is the use case for this and how would we classify a warning?

bhala-ki commented 1 year ago

Warning is a good status to be shown in the test report as it's like how it's showing in the phone battery life for example where the iPhone states Low Battery @ 20 %

sobi-ki commented 1 year ago

@prashant-ramcharan is some cases we might want to skip the test if a condition is not there.

For example, a dropdown is not available on test env due to not being available. We don't to fail the test for this.

Any chance of having skipped tests or ignored added to the report?

Thanks

prashant-ramcharan commented 1 year ago

Hi, Courgette does not control the state of your tests, it simply runs your Cucumber tests and then uses the test results to generate a report.

If you need to skip or ignore a test based on a some condition (ie. dropdown is not available), then you would do this in your Cucumber test and this would be reflected in the Courgette report.

Courgette does not have any influence on when a test passes or fails, this is owned by Cucumber.

Courgette already shows skipped tests in the report.

prashant-ramcharan commented 1 year ago

On another note:

For example, a dropdown is not available on test env due to not being available. We don't to fail the test for this.

If your test is expecting a dropdown that is not available, shouldn't your test fail because you were expecting a dropdown but your application didn't display it? This seems like you have a non deterministic test case.