Correctly expanding Scenario Outline params for pytest BDD (fixes #636)

[chasinglogic]

This fixes #636 by making each scenario outline param a separate param in the generated allure JSON.

Context

When using a Scenario Outline and pytest-bdd the parameters passed to the allure listener were of the incorrect type. The Parameter class in Allure expects the value field to be a String but since the callspec has _pytest_bdd_example which is a dictionary the value would be of that type. Which would serialise to JSON correctly but cause the following error when Allure generate attempted to deserialise it:

Could not read test result file targetreports/6557fe86-9aff-4b34-9198-2a61facbdff6-result.json
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.lang.String` from Object value (token `JsonToken.START_OBJECT`)
 at [Source: (sun.nio.ch.ChannelInputStream); line: 1, column: 621] (through reference chain: io.qameta.allure.model.TestResult["parameters"]->java.util.ArrayList[0]->io.qameta.allure.model.Parameter["value"])

This change expands the pytest bdd examples so they are each a parameter and brings it in line with other test frameworks and Allure behaviour.

Checklist

• [x] Sign Allure CLA
• [ ] Provide unit tests

[CLAassistant]

All committers have signed the CLA.

[chasinglogic]

Will write unit tests later today and am signing the CLA now.

[chasinglogic]

Example of bad JSON before this change:

{
  "name": "<market> market, the Company logo returns the user to the home page [UK]",
  "status": "passed",
  "steps": [
    {
      "name": "Given the market is UK",
      "status": "passed",
      "start": 1641383124791,
      "stop": 1641383135301
    },
    {
      "name": "And I am on the Website home page",
      "status": "passed",
      "start": 1641383135302,
      "stop": 1641383141638
    },
    {
      "name": "When I click the Company logo",
      "status": "passed",
      "start": 1641383141639,
      "stop": 1641383141811
    },
    {
      "name": "Then I am on the home page in the relevant market",
      "status": "passed",
      "start": 1641383141811,
      "stop": 1641383141814
    }
  ],
  "parameters": [
    {
      "name": "_pytest_bdd_example",
      "value": {
        "market": "UK"
      }
    }
  ],
  "start": 1641383124772,
  "stop": 1641383141814,
  "uuid": "1a3b4903-ffca-bc2b-6df5-84d4a29e1466",
  "historyId": "7af04ec143715c102e88898fbce93f5f",
  "fullName": "header/logo.feature:<market> market, the Company logo returns the user to the home page",
  "labels": [
    {
      "name": "host",
      "value": "Anonymouss-MacBook-Pro.local"
    },
    {
      "name": "thread",
      "value": "4474-MainThread"
    },
    {
      "name": "framework",
      "value": "pytest-bdd"
    },
    {
      "name": "language",
      "value": "cpython3"
    },
    {
      "name": "feature",
      "value": "Website logo header checks"
    }
  ]
}

Good JSON after this change:

{
  "name": "<market> market, desktop size, the header contains the desktop Basket/cart icon [UK]",
  "status": "passed",
  "steps": [
    {
      "name": "Given the market is UK",
      "status": "passed",
      "start": 1642070685922,
      "stop": 1642070691849
    },
    {
      "name": "And the desktop window size is desktop",
      "status": "passed",
      "start": 1642070691854,
      "stop": 1642070691957
    },
    {
      "name": "And I am on the Website home page",
      "status": "passed",
      "start": 1642070691958,
      "stop": 1642070695608
    },
    {
      "name": "Then the site header contains the Basket/cart icon",
      "status": "passed",
      "start": 1642070695609,
      "stop": 1642070695634
    }
  ],
  "parameters": [
    {
      "name": "market",
      "value": "UK"
    }
  ],
  "start": 1642070685913,
  "stop": 1642070695635,
  "uuid": "ea31ba2b-7648-f82f-558c-7ab596d19331",
  "historyId": "6d7ed49e4a7590d0d7c5f1a67e81a8bd",
  "fullName": "header/basket.feature:<market> market, desktop size, the header contains the desktop Basket/cart icon",
  "labels": [
    {
      "name": "host",
      "value": "the-enterprise.local"
    },
    {
      "name": "thread",
      "value": "65605-MainThread"
    },
    {
      "name": "framework",
      "value": "pytest-bdd"
    },
    {
      "name": "language",
      "value": "cpython3"
    },
    {
      "name": "feature",
      "value": "Website basket/cart icon header checks"
    }
  ]
}

[chasinglogic]

So I've been working through the broken tests but would like some confirmation that this solution will be acceptable before I spend much more time on this if that's possible?

[JagadeeshJayachandran]

Any update, please. We have updated our feature files as per pytest-bdd 5.0.0, we didnt realize about this allure reporting issue. This issue blocks us from using pytest-bdd 5.0.0

[ofirm93]

@skhomuti is there a problem with this bug fix? May someone re-trigger the checks? as this is a real issue which makes tests disappear from the report.

[JagadeeshJayachandran]

Can we please fix this code merge errors

[chasinglogic]

@aspenboy @JagadeeshJayachandran while I appreciate the reviews, AFAICT the maintainer of this repository is @sseliverstov and they haven't merged any PRs in quite a long time.

I will not be monitoring this PR for further updates.

[skhomuti]

@chasinglogic hi! Thank you for your contribution and sorry for a little bit inactive period. I'll check this PR and return back for you with feedback. 🙏

[sqandrew]

hello guys! It already has a workaround but having a fix would be really great.

Will that fix will be released this year? @sseliverstov @skhomuti what do you think?

thanks in advance