CUCUMBER: UNDEFINED state on step shown as PASSED

[Vovanchik]

Hello,

I am using cucumber-js framework which returns additionally undefined state of step. As result scenarios marked as PASSED in the report what is not correct.

Could you please show in the report also undefined state? Color: light blue, Name of state: either UNDEFINED or PENDING.

Here is the json_report_example.txt

[BogdanLivadariu]

@Vovanchik is this file the actual report that cucumber-js outputs ? or you have tweaked it. I'm asking this, because the structure of the file differs from what cucumber-java outputs ...

[BogdanLivadariu]

this is the structure of cucumber-java json reports

[
    {
        "id": "account-holder-withdraws-cash",
        "tags": [
            {
                "name": "@super",
                "line": 1
            }
        ],
        "description": "As a Account Holder\nI want to withdraw cash from an ATM\nSo that I can get money when the bank is closed",
        "name": "Account Holder withdraws cash",
        "keyword": "Feature",
        "line": 2,
        "elements": [
            {
                "description": "",
                "name": "Activate Credit Card",
                "keyword": "Background",
                "line": 7,
                "steps": [
                    {
                        "result": {
                            "duration": 107447000,
                            "status": "passed"
                        },
                        "name": "I have a new credit card",
                        "keyword": "Given ",
                        "line": 8,
                        "match": {
                            "location": "ATMScenario.I_have_a_new_credit_card()"
                        }
                    },
                    {
                        "result": {
                            "duration": 16000,
                            "status": "passed"
                        },
                        "name": "I confirm my pin number",
                        "keyword": "When ",
                        "line": 9,
                        "match": {
                            "location": "ATMScenario.I_confirm_my_pin_number()"
                        }
                    },
                    {
                        "result": {
                            "duration": 14000,
                            "status": "passed"
                        },
                        "name": "the card should be activated",
                        "keyword": "Then ",
                        "line": 10,
                        "match": {
                            "location": "ATMScenario.the_card_should_be_activated()"
                        }
                    }
                ],
                "type": "background"
            }
        ],
        "uri": "net/masterthought/example/ATM.feature"
    }
]

and this is what you have provided

{
  "name": "REQ#2 - Delete assessment",
  "id": "Assessment;req#2---delete-assessment",
  "line": 18,
  "keyword": "Scenario",
  "description": "",
  "type": "scenario",
  "tags": [
    {
      "name": "@PROD",
      "line": 17
    },
    {
      "name": "@PATIENT",
      "line": 17
    },
    {
      "name": "@ASSESMENTS",
      "line": 17
    }
  ],
  "steps": [
    {
      "name": "I am on the login page",
      "line": 19,
      "keyword": "Given ",
      "result": {
        "status": "passed",
        "duration": 1681118300
      },
      "match": {
        "location": "/data/build/branches/Registry-Trunk-UI/cucumber_js/tests-ui-js/src/test/features/step_definitions/assessments/assessments_steps.js:40"
      }
    },
    {
      "name": "I should login as \"administrator\" and \"123qweasd\" password",
      "line": 20,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "name": "I should open patient",
      "line": 21,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "name": "I should add \"Asthma Initiative\" assessment",
      "line": 22,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "name": "I should make completed assessment",
      "line": 23,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "name": "I should add \"Asthma Initiative\" assessment again",
      "line": 24,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "keyword": "After ",
      "hidden": true,
      "result": {
        "status": "passed",
        "duration": 84556816
      },
      "match": {
        "location": "/data/build/branches/Registry-Trunk-UI/cucumber_js/tests-ui-js/src/test/features/step_definitions/support/world.js:11"
      }
    }
  ]
}

[Vovanchik]

Hi @BogdanLivadariu,

Ops, It look's like I cut too much. :(

I am using grunt-cucmberjs plugin which producing json file.

Attaching full source in txt file.

test-report-json.txt

[BogdanLivadariu]

@Vovanchik by the looks of it this report does not has undefined steps or tags

can you attach a file that has undefined steps & tagged reports ?

[Vovanchik]

Ops... Hard working day.

Adding file with undefined steps.

What do you mean under tagged reports? I have only tagged scenarios. In my file it's shown as: "tags": [ { "name": "@PROD", "line": 3 }

test-report-json.txt

[Vovanchik]

Hi,

Any updates on my issue?

[BogdanLivadariu]

@Vovanchik I'm on vacation :) will try to fill in with updates next week, when I get back :D

[Vovanchik]

Have a nice vacation. Waiting for your results.

[Vovanchik]

For steps it look's very good.

If some of the steps are in undefined state can we marked Scenario as PASSED instead of FAILED as it is right now.

What I see there not status field for scenarios, so plugin decided which status to set.

What do you think?

[BogdanLivadariu]

@Vovanchik I guess I can add an option in the plugin config, that you can tick so that undefined steps are ignored, and in a scenario if you have undefined steps, scenario will be marked as passed

what do you think ?

[BogdanLivadariu]

please create a new issue to cover this case, as it is an enhancement :D

[Vovanchik]

Cool thank you ;)

[Vovanchik]

New issue #49 created.