feature request: Report command runtime

[stianlagstad]

Hi, and thanks again for this very useful package.

I'd like pytest-workflow to display somehow how long each test case takes. If I add something like --durations=0 --setup-show as pytest options, then I do get information about how long each assertion took, but that does not include the runtime of the command.

For example, take this pytest-workflow test:

- name: "Test dummy"
  command: bash script.sh
  files:
    - path: testfile.txt

Where script.sh looks like this:

#!/usr/bin/env bash

sleep 20
touch testfile.txt

If I run this with pytest --durations=0 --setup-show -vv, what I get is this:

============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/stian/repo
configfile: pytest.ini
plugins: workflow-2.1.0, hydra-core-1.3.2
collecting ...
collected 2 items

Test dummy:
        command:   bash tests/modules/test_dummy/script.sh
        directory: /tmp/pytest_workflow_q6ifdlfz/Test_dummy
        stdout:    /tmp/pytest_workflow_q6ifdlfz/Test_dummy/log.out
        stderr:    /tmp/pytest_workflow_q6ifdlfz/Test_dummy/log.err

<runs-for-a-while-here.......>

tests/modules/test_dummy/test.yml
        tests/modules/test_dummy/test.yml::Test dummy::exit code should be 0.
        tests/modules/test_dummy/test.yml::Test dummy::testfile.txt::should exist.All tests succeeded. Removing temporary directories and logs. Use '--kwd' or '--keep-workflow-wd' to disable this behaviour.

=============================================================================================== slowest durations ================================================================================================

(6 durations < 0.005s hidden.  Use -vv to show these durations.)
========================================================================================= 2 passed, 1 warning in 22.30s ==========================================================================================

I would like a report of some type that shows me that the test case "Test dummy" took 20.something seconds. Is this possible with pytest-workflow today?

My motivation for wanting this is to be able to separate slow tests and fast tests, so that I can run fast tests in pull request CI, and slow tests only after having merged changes in my codebase.

[rhpvorderman]

I agree that pytest behaviour is not very useful here, as the asserts are much quicker than the workflow . I believe it prints when a workflow is done. Instead it could print the duration instead of just "done". That should be trivial to implement. I will add this to the list.