search

olexale / bdd_widget_test

A BDD-style widget testing library
MIT License
101 stars 30 forks source link

Add reporter compatible with CucumberStudio #46

Closed davidTruta closed 10 months ago

davidTruta commented 1 year ago

First of all, thank you! In my opinion this is the best package for writing widget tests as well as integration tests using BDD in terms of organization, ease of use and speed. Great work!

I think a great improvement would be added support for cucumber compatible Reporters so we can upload the test results to CucumberStudio using scripts. Other similar packages have that, but they are not as organized or easy to work with in my opinion.

olexale commented 1 year ago

Thank you for the kind words!

I reviewed some of the popular BDD packages. It is easy for them to implement this reporting as they run tests themselves. The idea behind bdd_widget_test is to keep it just a "dumb" converter from Gherkin to Dart, so that the Dart lang ecosystem will take care of the rest (run tests, collect reports, etc).

I don't see a way to make bdd_widget_test generate such reports, but it may generate code that generates them. I see the value in such reports, but I need to investigate a bit more.

Thanks for the proposition!

davidTruta commented 11 months ago

I found an alternative, I will leave it here in case anyone else is searching for something similar.

I managed to get the desired behaviour by using this package along with junitreport and hiptest-publisher and a shell script.

The script does the following:

  1. adds required permissions before executing the tests using adb shell pm grant APP_ID PERMISSION
  2. exports the feature files from CucumberStudio into our project hiptest-publisher --config-file=integration_test/report/cucumber.conf --output-directory='integration_test/features' --test-run-id=TEST_RUN_ID --without=actionwords
  3. runs the tests and writes the xml report with the command flutter test -d DEVIDE_ID --flavor FLAVOR integration_test/features/app_test.dart --reporter json | tojunit >integration_test/report/TEST-report.xml
  4. pushes the report to CucumberStudio hiptest-publisher --config-file=integration_test/report/cucumber.conf --push integration_test/report/TEST-report.xml --test-run-id=TEST_RUN_ID --push-format junit
olexale commented 10 months ago

That looks like not an alternative, but a good way to do the job :) It is far better to leverage the existing mechanism (flutter test that generates a junit report) than to create a custom solution.

If anyone would be interested in the custom solution implementation, the hooks mechanism is available in 1.6.4, I guess it should be a good starting point.

I'm closing this ticket as there is a solution to the problem. If you believe that it is not good enough — feel free to re-open the issue.