add error_message for steps that have the status pending

markstein commented 3 months ago

🤔 What's changed?

By evaluating the runs, it is sometimes helpful to see the detailed error message even for steps that are pending.

⚡️ What's your motivation?

The error message is also output for steps that have the status pending #2392

🏷️ What kind of change is this?

:zap: New feature (non-breaking change which adds new behaviour)

♻️ Anything particular you want feedback on?

📋 Checklist:

[x] I agree to respect and uphold the Cucumber Community Code of Conduct
[x] I've changed the behaviour of the code
- [ ] I have added/updated tests to cover my changes.
[ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
[x] Users should know about my change
- [x] I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

This text was originally generated from a template, then edited by hand. You can modify the template here.

coveralls commented 3 months ago

coverage: 98.392%. remained the same when pulling 5935167cb8fcdeea7f7270796dd6ab1a81f5c16d on markstein:main into 5df8971b3ba0cc12bfc1e9e4e7b1599622c25e9b on cucumber:main.

davidjgoss commented 3 months ago

I'm also curious as to what a "detailed error message" would be in case of pending, given it is supposed to just mean the step isn't implemented yet.

markstein commented 3 months ago

@davidjgoss thanks for taking the time for the review.

yes, we use it as a callback variant in the afterStep hook. We use the pending status partly for integration tests that are currently "unstable" due to instability in external systems. However, other sources of error could possibly arise, which would then be recognized based on the error message.

Furthermore, if you look at the Java implementation (which is not necessarily relevant), but the behavior reflects: https://github.com/cucumber/cucumber-jvm/blob/6549826da802dbf031b89590d2cf4bbdeb36fc3c/cucumber-core/src/main/java/io/cucumber/core/plugin/JsonFormatter.java#L317

Based on json result, the error message is then displayed in the html report.

davidjgoss commented 2 months ago

Released in https://github.com/cucumber/cucumber-js/releases/tag/v10.5.0

cucumber / cucumber-js