search

pytest-dev / pytest-bdd

BDD library for the py.test runner
https://pytest-bdd.readthedocs.io/en/latest/
MIT License
1.3k stars 219 forks source link

Using frame inspection to obtain feature file path is unreliable #401

Open DrewDevereux opened 3 years ago

DrewDevereux commented 3 years ago

Pytest-bdd uses frame inspection to obtain the path to its feature file. This in unreliable because

Pytest performs assert rewriting, and caches the rewritten modules. If I run pytest-bdd on my local system, where my tests are in "/projects/my-project/tests", then the cached rewritten modules will end up containing this path in their metadata.

If I then spin up a docker container, with my project mounted as "/app/my-project", then when I run my tests, pytest will load the cached rewritten modules, and the metadata will be wrong. This usually doesn't matter, but it matters to pytest-bdd, because pytest-bdd will get the wrong path, and go looking for its feature file in "/projects/my-project/tests/....", and crash with a FileNotFoundError.

DrewDevereux commented 3 years ago

Workaround for the docker case is to have **/__pycache__ in your .dockerignore. But this doesn't fix the general issue of running locally or across a mount.

DrewDevereux commented 3 years ago

I am continuing to run into this issue when deploying the same test suite to different docker containers.

The above workaround is fine if you copy the test suite into your image at build stage. It doesn't help if you mount the test suite at container runtime.

For runtime, a workaround is always to run your tests with PYTHONDONTWRITEBYTECODE environment variable set to 1, or using -B option to python i.e. python -B -m pytest.