search

smarie / python-pytest-steps

A tiny package to ease the creation of test steps with shared intermediate results/state.
https://smarie.github.io/python-pytest-steps/
BSD 3-Clause "New" or "Revised" License
57 stars 5 forks source link

yield <step-name> does not work without extra yield at the top of the test case #53

Open danielFlemstrom opened 2 years ago

danielFlemstrom commented 2 years ago

Trying out the example from documentation python-pytest-steps/, a hint is given to add the step name to yield. The code on the documentation shown below needs to be complemented with an extra yield (line 13) in the following image. image

danielFlemstrom commented 2 years ago

Digging deeper into this matter, it looks like the steps are handled differently inside VSCODE and when run on the commandline.

When running the above snippet in VSCODE an extra yield at the top works fine although the steps are enumerated in random order in the drop down menu of VSCODE. I extended the test case from the documentation:

image

Starting the test case in VSCODE, they are started like this without the first yield

> ./.venv/bin/python -m pytest --rootdir . --override-ini junit_family=xunit1 --junit-xml=/var/folders/pt/zw8ycd8x1pl7w13s9ltgrvlc0000gq/T/tmp-361377bjf1ByIrKIL.xml ./tests/integration/apis/test_user_files.py::test_suite[4.step_d] cwd: . I.e. one line per step, repeated In the order d,e,b,a,c.

And fails with: ´./tests/integration/apis/test_user_files.py::test_suite[3.step_c] Failed: [undefined]pytest_steps.steps_generator.StepYieldError: Error collecting results from step '3.step_c': received '1.step_a' from the yield statement, which is different from the current step or step name. Please either use yield, yield '3.step_c' or wrap your step with with optional_step(...) as my_step: and use yield my_step´

Running the same test case from the command line with ´pytest´ works fine though.

This is actually a huge problem for us using VSCODE and a CI/CD pipeline. Any suggestions or ideas are more than welcome!

smarie commented 1 year ago

Hi @danielFlemstrom, thanks for reporting ! It seems indeed very much related to VSCode... Proably the best and fastest workaround is to get rid of pytest steps and instead passing a session-scoped fixture to all of your "steps" tests. You can then further use a basic test class to wrap the various "step" tests, if you want to have them nicely separated from others.

That's the easiest workaround I would suggest, not requiring strong investigations on how VSCode inspects pytest (which I do not have the bandwidth to perform as of now unfortunately)

I hope this helps!