integration with jira and xray

[ckstettler]

We are looking to leverage godog for our integration testing. We also are looking leverage the xray plugin in jira. xray supports importing of test results from cucumber or xray formats.

http://confluence.xpand-addons.com/display/XRAY/Import+Execution+Results+-+REST

Any plans to support additional formats?

[l3pp4rd]

Well, I personally do not use any other formats, so there are no plans for such from my point of view. but if a contribution is made for such a format, then it can be accepted. you may also write a package with this format and register it as godog.Format by following a sample like junit

[ckstettler]

I have started doing exactly what you have suggested. Hopefully you will see a PR soon. Thank you for the quick response.

[l3pp4rd]

OK then, thanks

[ckstettler]

I have been looking at a cucumber example at https://www.relishapp.com/cucumber/cucumber/docs/formatters/json-output-formatter. The sample ruby step definitions show two commands "embed" and "put" which cucumber understands and writes to the test output file.

Examples: embed "screenshot.png", "image/png" puts "from step definition"

Does godog support these? I dont think I see these supported but wanted to confirm first. I they are not then I can ignore them for now in the cucumber formatter.

[l3pp4rd]

currently not supported in godog.

[ckstettler]

Before I get too far down my testing and then find you dont like the approach. I have created a feature file to test the cucumber formatting. I leveraged two existing steps in suite_test.go and created one additional whose purpose is to validate the json output.

To validate the json I chose to put the expected json string as a docstring vs creating steps to validate json fields independently. The docstring approach seemed to make it easy. I generated the docstring by running a ruby cucumber and getting the json output for the same gherkin that I have defined in the "Given" step.

Is this approach good for you? Or do you have some other recommendation?

formatter/cucumber.feature:

Feature: cucumber json formatter In order to support tools that import cucumber json output I need to be able to support cucumber json formatted output

Scenario: Support of Feature Plus Scenario Node Given a feature "features/simple.feature" file: """ Feature: simple feature simple description Scenario: simple scenario simple feature description """ When I run feature suite with formatter "cucumber" Then the rendered json will be as follows: """ application/json [ { "uri": "features/simple.feature", "id": "simple-feature", "keyword": "Feature", "name": "simple feature", "description": " simple description", "line": 1, "elements": [ { "id": "simple-feature;simple-scenario", "keyword": "Scenario", "name": "simple scenario", "description": " simple feature description", "line": 3, "type": "scenario" } ] } ] """

[l3pp4rd]

Hi @ckstettler yes that makes sense, I would have done it the same way. Thanks for asking to confirm, good work!

[ckstettler]

One other knit I found that the godog data model seems to only have comments at the Feature level. I was not able to see comments attached to Scenarios or Step data structures. I dont see this as a major issue but it was different that cucumber unless I am missing something...

Also regarding the docstring, is step.Argument always a nil value or *gherkin.Docstring? Or can it be of other types? I put logic to test to see if it was a docstring but wasnt sure if that was necessary. I am probably safer checking the assertion status regardless, however, I am curious?

I think the code is complete. I need to do an internal review on my side to get approval to submit it, but it should be in the next day or two.

[l3pp4rd]

godog is shipping gherkin package together, but it acts as vendor package, currently packaged with gherkin v3. even the master branch, does not support the comments in gherkin, see scenario def.

Arguments can be:

1. any golang primitive
2. *DocString
3. *DataTable

Maybe you could add a comment somewhere in the code (where you mentioned embedded error details like images or stack traces) this may be added to godog later, as it might have a different type of error which could support context.