search

eugene-sea / karma-cucumber-js

Karma adapter for running Cucumber.js features
10 stars 4 forks source link

What Reports Should I Be Generating? #7

Closed JimLynchCodes closed 8 years ago

JimLynchCodes commented 8 years ago

Hi, I currently have karma set up with Istanbul code coverage tool. This is great as we like to see what % of our codebase is covered by unit tests. We are mostly using 'protractor-cucumber-framework' to run gherkin scenarios with ui / web tests in the step definitions. However, we would like to implement some of our step definitions as unit tests, and for that it's really awesome to have cucumber reports generated. It is possible to have both types of reports generated? Does it make sense to still generate code coverage from you step definitions? Should I have cucumber-karma tests and regular karma unit tests or can I all-out switch over to cucumber-karma tests? Awesome project. I'm just trying to figure out the best way to fit it into our development process. Thanks, Jim

eugene-sea commented 8 years ago

Hi,

In my projects I use both BDD tests and regular unit-tests for back-end and front-end. Thus, yes I suggest to not kill your regular karma tests. BDD specs should contain only key scenarios written in business language. Often this is not enough for complete testing: you may need to test additional scenarios or test things related to implementation. Regular unit-tests are needed for this.

Code coverage from BDD tests may be useful if combined with code coverage from other test types. (I have not done this yet.)

In my projects the most useful "report" is "living documentation site". It looks like this.

It is used by all parties (business, technical, non-technical) involved in projects to discover current state of business requirements.

JimLynchCodes commented 8 years ago

Hey!

Thanks for getting back to me. What you're saying makes sense, and it's pretty consistent with what I ended up doing. For functional UI stuff we use protractor-cucumber-framework. Then for scenarios that are better implemented with karma / unit-test syntax we use this framework. BUT, sometimes we want to just add unit tests without adding "boring" business cases, so we keep regular karma tests as well. I was worried about the fragmented reports (specifically the code coverage report), but I found instanbul-combine to combine the coverage reports and CombineCucumberJSON for the cucumber reports.