How to run pytest-workflow from Python?

[stevekm]

I don't want to use a yaml file in all cases to run pytest-workflow. But I am not seeing any docs anywhere that show how to run it directly from Python. Are there any? Thanks.

[rhpvorderman]

You can run custom tests from python: https://pytest-workflow.readthedocs.io/en/stable/#writing-custom-tests

But indeed, that still requires defining the workflow in yaml. It is currently not possible to define a workflow in python.

Thinking about it, the YAML is actually the most defining feature of pytest-workflow. If you don't need it, won't using a subprocess call in a global scoped fixture achieve the same thing in pytest? That way there is no need to use pytest-workflow at all and you can happily hack away in Python.

[stevekm]

well that is basically what I have already got; https://github.com/mskcc/pluto/blob/master/plutoTestCase.py

However I still have issues with large amounts of code duplication for my test cases; https://github.com/mskcc/pluto-cwl/blob/dev/tests/test_samples_fillout_index_batch_workflow_cwl.py

I like the feature set offered in pytest-workflow but I still need a lot of my own custom methods for various tasks, so I am not yet ready to switch 100% to yaml for everything. In particular, I am worried about potential issues with dynamically adjusting the CLI commands for running the workflows at runtime; https://github.com/mskcc/pluto/blob/master/settings.py . Also have a pretty large amount of custom assertions I would like to keep available; https://github.com/mskcc/pluto/blob/master/plutoTestCase.py#L366-L702

Was mostly hoping to start out by using the library from the python side before trying to migrate to yaml

[rhpvorderman]

I guess you could try creating a small pytest plugin yourself that uses WorkflowTestsCollector https://github.com/LUMC/pytest-workflow/blob/develop/src/pytest_workflow/plugin.py#L349. But that would require diving a bit deeper into pytest's API and I don't know if it is possible to achieve the desirable results that way. The plugin is quite YAML oriented as it assumes that the datastructure that the workflowtestscollector gets is ordered in just the same way as the yaml file. So that might be a hindrance on that path.

like the feature set offered in pytest-workflow but I still need a lot of my own custom methods for various tasks, [...] Was mostly hoping to start out by using the library from the python side before trying to migrate to yaml

I can see where you are coming from. Pytest-workflow was never intended to replace custom tests though. It has always been intended to make it easier to create the simple 90% of the test cases where you just want to see if the workflow succeeds and creates the correct files. Parametrizing workflow runs with all kinds of different settings was never considered. Primarily because workflows run a long time and creating a matrix of 64 possible workflows was never preferred over selecting the 5 most common use cases in our testing.

The easiest way to interact with pytest-workflow is going trough the YAML, though I do know of people who have written a custom framework to create the pytest-workflow YAML files for them automatically in their own DSL..

[timini]

from pytest_workflow.workflow import Workflow