Background steps are reported without keyword Background in json report

usertest2024 commented 6 months ago

👓 What did you see?

In json report, steps coming from Background section are not identified as background steps. So, the tool we are using internally to generate custom report, is not working any more.(cannot find keyword: Background) I have in json report:

                "steps": [

                    {
                        "arguments": [],
                        "keyword": "Given ",
                        "line": 8,
                        "name": "this is a background step",
                        "match": {
                            "location": "src/impl/steps/stepDefinition.steps.ts:25"
                        },
                        "result": {
                            "status": "passed",
                            "duration": 1610877746
                        }
                    },

✅ What did you expect to see?

I expect to see something like:

      "elements":[
         {
            "type":"background",
            "keyword":"Background",
            "steps":[
               {
                  "result":{
                     "duration":1390492,
                     "status":"passed"
                  },
                  "line":9,
                  "name":"this is a background step",
                  "keyword":"Given "
               }
            ]
         },
......

📦 Which tool/library version are you using?

"@cucumber/cucumber": "^10.3.1"

🔬 How could we reproduce it?

No response

📚 Any additional context?

No response

davidjgoss commented 6 months ago

So, the tool we are using internally to generate custom report, is not working any more.

This JSON output has been in maintenance mode for some time. Have you been using another version of cucumber-js that produces for format you expect from the JSON formatter? If so, can you say which version?

usertest2024 commented 6 months ago

Hello, we used our tool with java projects (cucumber-java 7.14.0). In Json outputs, background steps are correctly identified .

      {
      "type":"background",
       "keyword":"Background",
        "steps":[
      {
        "result":{
        "duration":1390492,
        "status":"passed"
        },
        "line":9,
        "name":"this is a background step",
        "keyword":"Given "
        }
        ]
        }

Now, we are automating front projects and we use cucumber": "10.3.1".

In generated Json report, background steps are not flagged with keyword Background as per cucumber-java.

   {
                "arguments": [],
                "keyword": "Given ",
                "line": 8,
                "name": "this is a background step",
                "match": {
                    "location": "src/impl/steps/stepDefinition.steps.ts:25"
                },
                "result": {
                    "status": "passed",
                    "duration": 1610877746
                }
            }

Hope my explanation is clear for you Thank you for your help

davidjgoss commented 5 months ago

Thanks, that makes sense now. Let me come back to you shortly on this.

mpkorstanje commented 5 months ago

@usertest2024 unfortunately the JSON output was developed before things like JSON Schema got popular and so different Cucumber implementations have come to differ over time.

It is worth looking at test data from cucumber-js and cucumber-jvm in the cucumber-json-converter:

For cucumber-jvm backgrounds are represented as their own unique element with "type": "background" in the elements array. Distinct from the scenario elements. Where as in cucumber-js, only elements of "type": "scenario" are used, and the background steps are folded into the scenario. Changing this seem like a breaking change of the cucumber-js flavor of the json to me, but I'll defer to @davidjgoss for judgement on that.

While we are slowly working on a common output format with a schema, you may be better of fixing your custom reporting tool to be lenient enough to accept both formats. The samples in the testdata folder should be able to help you with that.

usertest2024 commented 5 months ago

@mpkorstanje thank you for the answer and the explanations Unfortunately , for documentation purposes and regulatory compliance, we need to include the background in the generated reports as 'initial conditions' seperate section and not in scenario descriptions. This is a mandatory section required to pass the audit of our documents."

mpkorstanje commented 5 months ago

Then I am afraid you will have the unenviable task of reimplementing the json formatter from cucumber-jvm in javascript.

Or perhaps a less involved solution would be to try and post process the json report and convert it to the jvm flavour. The uri field provides the file location. You can parse it with the Gherkin parser. Then by matching the steps in the report to the steps in backgrounds of the document you can split them.

Finally, we are (and have been for a long time) planning to rewrite the json formatter used by cucumber-jvm to use the message protocol - similar to the junit-xml-formatter. This should allow you to consume the message protocol from cucumber-js and render that as a json report. But the timeline on this is indefinite, so I don't think this will meet your needs.

github-actions[bot] commented 5 months ago

This issue is stale because it has been open for 3 weeks with no activity. Remove the stale label or comment or this will be closed in another 5 days.

github-actions[bot] commented 5 months ago

This issue was closed because it has been stalled for 5 days with no activity.

davidjgoss commented 4 months ago

Sorry for the bot-triggered closure here. But as per @mpkorstanje earlier comments, this would be a breaking change if we were to make it, and with this formatter a) in maintenance mode and b) widely relied upon that's not something we can do unfortunately. Post-processing the JSON as suggested is probably the way to go.

cucumber / cucumber-js