search

pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.95k stars 2.66k forks source link

Add hooks to allow further AST customization and control pyc caching while assertion rewriting. #12387

Open jaltmayerpizzorno opened 4 months ago

jaltmayerpizzorno commented 4 months ago

What's the problem this feature will solve?

SlipCover and soon also Coverage.py modify the AST while loading modules, including test modules, to facilitate measuring branch coverage. However, pytest's assertion rewriting loader overrides that mechanism.

Describe the solution you'd like

I suggest that a hook be added to allow plugins to further modify the AST before it is compiled (or, alternatively, provide an already-modified AST for pytest to assertion rewrite and compile).

Because the modified AST leads to bytecode that only makes sense when also measuring coverage, and in some cases is run specific, I suggest that hooks also be added that allows the plugin to either prevent pyc caching or add to its file name, so that it indicates it has been modified for use with a certain SlipCover and/or Coverage.py version.

Alternative Solutions

I have a workaround for SlipCover that dynamically modifies _pytest.assertion.rewrite to insert hooks, but that could break as that module is modified, making it hard to maintain.

Additional context

I described the mechanism used by SlipCover, and which @nedbat has been working on integrating into Coverage.py, in my recent PyCon'24 talk. I'd be happy to provide further details, or even submit a PR.

Zac-HD commented 4 months ago

I'd be happy to consider a detailed proposal for new hooks here, and expect to support a following PR 😁

Quick selection of references: Hooks reference, how-to write hooks (for existing hooks), assertion rewriting docs. We'll need to make sure that the new hook(s) play(s) nicely with our existing functionality around assertion rewriting, and works smoothly if multiple plugins implement such a hook.