Inline JSON Report

[smarlowucf]

I created a PR in the pytest-json project which would allow the use of the plugin from code rather than saving to a file. Since that project is not maintained would there be interest and adding similar functionality to pytest-json-report?

This would be helpful when running pytest in code:

cmds = shlex.split(args)
plugin = JSONReport()
result = pytest.main(cmds, plugins=[plugin])
plugin.report # report dict

I can make a PR but wanted to gauge interest first.

[numirias]

Thanks for suggesting! Being able to access the JSON report directly from code is definitely useful.

Currently you can take advantage of the provided hooks to access the report, e.g. in conftest.py or in a mocked-up plugin:

class MyPlugin:

    def pytest_json_modifyreport(self, json_report):
        ... # Do something with json_report

pytest.main(['--json-report'], plugins=[MyPlugin()])

Or:

def pytest_sessionfinish(self, session):
    report = session.config._json_report.report

What would be the advantage of handling the JSONReport() object directly as in your snippet? From looking at pytest-html and others, most of them don't seem to let you initialize the plugin yourself. Is there a technical advantage?

[smarlowucf]

Is there a technical advantage?

The specific use case is here https://github.com/SUSE/ipa/blob/master/ipa/ipa_provider.py#L295. And leverages a very simplified plugin https://github.com/SUSE/ipa/blob/master/ipa/results_plugin.py.

The goal to run multiple pytest sessions and aggregate the results into a dictionary without having to write/read from disk. Would like to leverage a more robust plugin for this but not sure a better way to access the results outside the realm of conftest.py hooks.

So from your first example I could do something like:

class MyPlugin:
    def pytest_json_modifyreport(self, json_report):
        self.report = json_report

plugin = MyPlugin()
pytest.main(['--json-report'], plugins=[plugin])
plugin.report

If so is it possible to prevent the pytest-json-report plugin from also writing the report to disk? Since the goal is to aggregate many session reports before persisting.

[numirias]

I was concerned it would be nasty to get a handle to pytest.config when using the plugin directly from code, but it's surprisingly straightforward. With the changes in #12 you should be able to use the plugin the way you planned.

If so is it possible to prevent the pytest-json-report plugin from also writing the report to disk?

If you want I can add a switch, but writing to /dev/null seems conventional:

import os

pytest.main(['--json-report', '--json-report-file', os.devnull], plugins=[plugin])

If that looks good to you, I'll merge the PR and make a new release.

[smarlowucf]

If that looks good to you, I'll merge the PR and make a new release.

Yeah, that looks great, thanks! I think writing to /dev/null is sufficient.