search

JamesHutchison / pytest-hot-reloading

A hot reloading pytest daemon, implemented as a plugin
MIT License
86 stars 2 forks source link
hot-reload pytest pytest-plugin python testing

readme

A PyTest Hot Reloading Plugin

versions

A hot reloading pytest daemon, implemented as a plugin.

Notice - Busted VS Code behavior - 5/28/2024

It was noted that VS Code does not properly detect tests finishing. This is due to buggy experimental behavior that you opt-in to. To opt-out, add to your settings:

"python.experiments.optOutFrom": [
    "pythonTestAdapter"
],

Features

Trade-offs

If it takes less than 5 seconds to do all of the imports necessary to run a unit test, then you probably don't need this.

If you're using Django, recommended to use --keep-db to preserve the test database.

The minimum Python version is 3.10

Demo

With the hot reloading daemon

Faster subsequent runs

Hot reloading demo

Without the hot reloading daemon

Not hot reloading demo

Installation

Do not install in production code. This is exclusively for the developer environment.

pip: Add pytest-hot-reloading to your dev-requirements.txt file and pip install -r dev-requirements.txt

poetry: poetry add --group=dev pytest-hot-reloading

Usage

Add the plugin to the pytest arguments.

Example using pyproject.toml:

[tool.pytest.ini_options]
addopts = "-p pytest_hot_reloading.plugin"

When running pytest, the plugin will detect whether the daemon is running. If the daemon is not running, it will error unless the --daemon-start-if-needed argument is passed.

The current version of the VS Code Python extension is not, by default, compatible with automatically starting the daemon. The test runner will hang. However, you can revert to legacy behavior which will allow for using the automatic starting. See the VS Code section below for more information.

The recommended way to run the daemon is to give it its own run run profile so you can easily use the debugger in tests. As a convenience you can also, if you're using a dev container, add this to your postStartCommand: pytest --daemon &. If the daemon doesn't start from your postStartCommand, see: https://github.com/microsoft/vscode-remote-release/issues/8536

Note that a pid file is created to track the pid.

Imports and in many cases initialization logic are not reran on subsequent runs, which can be a huge time saver.

Currently, if you want to debug, you will want to run the daemon manually with debugging.

JetBrains (IDEA, PyCharm, etc)

Create a REGULAR Python run configuration, with pytest as the module. For parameters, add --daemon. Strongly consider storing in the project so it is shared with other developers. Note that you most likely also need to set the working directory to the project root where the pytest configuration is located so that it knows to use the plugin you configured earlier.

JetBrains Example

For more information on parameters, see the VS Code section below.

VS Code

Debugging demo

This can easily be done in VS Code with the following launch profile:

        {
            "name": "Pytest Daemon",
            "type": "python",
            "request": "launch",
            "module": "pytest",
            "justMyCode": false,
            "args": [
                "--daemon",
                //
                // everything below this is optional
                //
                "--daemon-port",
                "4852", // the default value
                "--daemon-watch-globs",
                "./*.py" // the default value
                // "./my-project/*.py:./some-thing-else/*.py",  // example of colon separated globs
                "--daemon-ignore-watch-globs",
                "./.venv/*" // this is the default value, also colon separated globs
            ]
        },

The daemon can be configured to use either file system polling or OS-based file system events. The polling behavior is used by default and has higher compatibility. For example, if you're using Docker for Windows with WSL2, you're going to have a bad time with inotify.

If the daemon is already running and you run pytest with --daemon, then the old one will be stopped and a new one will be started. Note that pytest --daemon is NOT how you run tests. It is only used to start the daemon.

The daemon can be stopped with pytest --stop-daemon. This can be used if it gets into a bad state.

To enable automatically starting the server, you have to, currently, disable the new Python Test Adapter:

In your devcontainer.json or user settings:

"python.experiments.optOutFrom": [
    "pythonTestAdapter"
],

Then enable automatically starting the daemon in your settings:

"python.testing.pytestArgs": [
    "--daemon-start-if-needed",
    "tests"
],

Arguments and Env Variables

Workarounds

Libraries that use mutated globals may need a workaround to work with this plugin. The preferred route is to have the library update its code to not mutate globals in a test environment, or to restore them after a test suite has ran. In some cases, that isn't possible, usually because the person with the problem doesn't own the library and can't wait around for a fix.

To register a workaround, create a function that is decorated by the pytest_hot_reloading.workaround.register_workaround decorator. It may optionally yield. If it does, then code after the yield is executed after the test suite has ran.

Example:

from pytest_hot_reloading.workaround import register_workaround

@register_workaround("my_library")
def my_library_workaround():
    import my_library

    yield

    my_library.some_global = BackToOriginalValue()

If you are a library author, you can disable any workarounds for your library by creating an empty module _clear_hot_reload_workarounds.py. If this is successfully imported, then workarounds for the given module will not be executed.

Need More Speed?

Known Issues

Notes