Unable to run pytest-xdist with pytest-bdd-html

[AmirYa412]

I am unable to run pytest-bdd-html with pytest-xdist.

I reported the issue in pytest-bdd git, and after a suggestion by the dev to remove pytest-bdd-html dependency, the issue is fixed. I am able to run the plugin with pytest-parallel but pytest-parallel is seems to be an outdated library that is no longer supported and lacking options that pytest-xdist has.

Stack:

pytest==7.4.3
pytest-bdd==6.1.1
pytest-bdd-html==0.1.14a0
pytest-html==3.2.0
pytest-metadata==3.0.0
pytest-xdist==3.5.0     
requests==2.31.0
urllib3==1.26.12

Error:

bdd/lib/python3.7/site-packages/execnet/gateway_base.py", line 1503, in _save
INTERNALERROR> E                 raise DumpError(f"can't serialize {tp}")
INTERNALERROR> E             execnet.gateway_base.DumpError: can't serialize <class 'pytest_bdd.parser.Step'>
INTERNALERROR> E           assert False

[slavos1]

Hello @AmirYa412, thanks for using pytest-bdd-html and for your detailed report. The issue is known (reported before in #6 and #18). I will close this a duplicate of #6 so that it is tracked in only one place.

[AmirYa412]

@slavos1 thanks for the response! Waiting forward for a fix!

P.S I can share feedback from a big project using your plugin, if you look for ideas on how to improve it further! Thanks again.

[slavos1]

if you look for ideas

Great, I would love that. Feedback, ideas and PRs are always welcome! :)

[AmirYa412]

@slavos1 Sure, writing them here(hope that's ok):

1. Printing created fixtures.

Consider adding fixtures created by the scenario test node to the report stdout, I did it by simply printing them in conftest.py. This helped me massively to debug failed tests, as I can see all the relevant data created throughout the scenario execution.

# conftest.py
def pytest_bdd_after_scenario(request):
    # Add relevant scenario's fixtures to report's stdout
    excluded_fixtures = ("pytestconfig", "request")
    try:
        for fixture_name in request.node.fixturenames:
            if fixture_name not in excluded_fixtures:
                fixture_value = request.getfixturevalue(fixture_name)
                print(f"{fixture_name}: {fixture_value} \n")
    except Exception as e:
        print(f"Error adding fixture data to stdout: {e}")

2. Better indication for step's failure.

When a scenario is using the same step multiple times, and such step fails, its hard to understand which one of them has failed. I suggest editing the scenario description and highlight the failed step.

For example, imagine you have this scenario:

Scenario: Do simple get request
    Given auth as free user
    When get request
    Given auth as paid user
    When get request

If the scenario will fail on When get request its really hard to understand, which one was it.

I went for different approach to solve it as suggest above, by using these hooks:

a. Changed "Links" column title to "Failed Step", as I don't really need "Links":

def pytest_html_results_table_header(cells):
    try:
        # Rename Links column to "Failed Step"
        cells[-1] = html.th("Failed Step", class_="sortable result-links", col="result-links")
    except Exception:
        pass

b. Add the failed step with it's index to the same column per scenario:

def pytest_html_results_table_row(report, cells):
    try:
        # Add step name to Failed Step cell
        if report.failed:
            steps = report.scenario["steps"]
            failed_step = next(step for step in steps if step["failed"])
            step_name = failed_step['name']
            step_position = steps.index(failed_step) + 1
            step_keyword = failed_step["keyword"]
            cells[-1] = html.td(f"[{step_position}] {step_keyword} {step_name}", class_="col-name",
                                style="color:red; font-weight:bold; font-size:15px")
    except Exception:
        pass

While this solution makes me count manually the index of step, it solved the problem.

Hope this helps, this plugin is great.

[slavos1]

Lots to read, great! Let me read thought it. Thanks.

[slavos1]

Hello @AmirYa412, Thanks again for your feedback and ideas and time you spent reproducing.

1. Printing created fixtures.

Oh, took me some time to get it -- you want to see the value of each fixture ... I understand the need, just maybe the output can end up noisy for some use cases (when there's lots of data transferred between steps). If I am to add such names/values to the Description column instead of stdout, even the short strings may break steps display. I'd therefore skip on this suggestion.

2. Better indication for step's failure.

A good idea indeed -- can be simplified by directly wrapping each failing step in an "error" span, for example, so that you can see the failed step as, say, "red"+bold; and no need to change/rename other column(s).

And not to forget:

Hope this helps, this plugin is great.

Appreciate -- exactly what I wished for, that someone will find my code useful. 👍🏻

[slavos1]

Will reference in a new issue.

[AmirYa412]

Thanks @slavos1.

Regarding 1 I've meant to add them to the stdout, not to the description column. It doesn't hurt to have them, and user will need to expand the test report to see it. For me, it was crucial - so I can check IDs and such generated during test run, but I understand if you will skip it.

See example, user won't see it in report, unless clicking on the test row to expand:

Regarding 2 This approach sounds great, looking forward to see it!

[slavos1]

Hi @AmirYa412, re 1: having it "hidden" in stdout is ok, I am just not sure it is specific enough for this repo. What I mean is it's a nice tip but you can use it regardless if you are using pytest-bdd-html or not.

Re #20 -- thanks again for the tip, I have planned it for later.

[AmirYa412]

@slavos1 you are right, probably suited more to pytest-html. I just find the stdout for pytest-bdd reporting is extremely lacking. Thanks!