Can no longer add images to report during teardown phase

[zolveuran]

Hello,

I have noticed that adding images to the report during the teardown phase no longer works in pytest-html version 4.1.1. It used to work in version 3 though. I've added a minimal example of a conftest.py below. Setting report.when == "teardown" to "call" instead makes the plots appear.

I need this behavior as I'm using pytest for hardware in the loop testing. My fixtures represent actual lab equipment that disconnect and save all data to file in their teardown phase. In my actual pytest_runtest_makereport() I discover all generated plots and put them in the report under the correct test case.

Versions where adding images in teardown works:

• pytest==7.4.3
• pytest-html==3.2.0

Versions where adding images in teardown doesn't work:

• pytest==7.4.3
• pytest-html==4.1.1

import pytest
import pytest_html

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    """Populates description column in report and adds images."""
    outcome = yield  # Get test results
    report = outcome.get_result()
    # Add description column with test function docstring
    report.description = str(item.function.__doc__)

    extras = getattr(report, "extras", [])

    # Generate and add plots after pytest fixture teardown
    if report.when == "teardown":

        plot_path = "./some_plot.png"
        extras.append(pytest_html.extras.png(plot_path))
        report.extras = extras

Image of report when plotting in "call":

Image of report when plotting in "teadown":

If this is fixed I also propose that a mention of changing the report.when == "call" line is put in the documentation. It was super important to me to be able to change this, but it took a lot of staring to realize why plotting didn't work as expected. Took me a week to notice that the plots put into the report where from the previous run when adding them during "call", since plots were not generated by fixtures before "teardown".

Otherwise a super happy user of pytest-html! It has really been a godsend to generate a report with logs and plots for 150 test cases. Great work!

[BeyondEvil]

Thanks for reporting!

I will look into it.

However, I'm not exactly clear where in the documentation you want a fix? 🤔

[zolveuran]

Thanks for the quick reply!

Regarding where to update the documentation, I think a single sentence after the pytest_runtest_makereport example would do the trick. Something like:

You can use report.when to specify when during the test run the extras will be added to the report. Allowed values are setup, calland teardown.

It could come directly before the name argument example. There might be more allowed values for report.when, but these I could name on the top of my head.

I also notice that extras can be added via fixtures now. I would hazard a guess (haven't tried) that one could use this fixture to make pytest wait for some other fixture to be torn down first. However, specifying report.when was simple and did the trick.

[robsan00]

I think I have got the same issue. I try to add URL links that don't work anymore with >=4.0.0 and when printing out "report.when", I am in the teardown (as I am doing this during post-check of the test to check some state of the device I am testing after running the test). Besides using "url" instead of "png" and not doing any explicit "when" check, my code looks about the same.

[yugokato]

@BeyondEvil Do you know when this fix will be actually released? I'm assuming it will be in the next release but I'm wondering when it will be as it's been 5 months. I'm needing this to work to migrate my project to version 4.x.

[zolveuran]

@BeyondEvil Also curious on when we can get this. Could it be released as v4.1.2 now? I imagine new (and supposedly old) users to pytest-html might be confused by report.when not working as intended