search

pytest-dev / pytest-forked

extracted --boxed from pytest-xdist to ensure backward compat
MIT License
62 stars 21 forks source link

Useful against pollution of interpreter state #8

Open asmodehn opened 6 years ago

asmodehn commented 6 years ago

Just wanted to point that this package can be a useful protection against "pollution" of the interpreter state.

For example when testing behaviour of custom imports, given the global state of sys, and the cache mechanisms, having boxed processes prevent unknowingly reusing a module imported from another test. And there might be more cases like this one.

We should probably gather them in one place (README ?) as they might be a bit esoteric for the newcomer, and it would be useful to pin them down.

RonnyPfannschmidt commented 6 years ago

feel free to do that, also please note that this plugin is currently unmaintained

untitaker commented 4 years ago

I believe pytest-forked needs to develop some patterns or best practices around it. Most importantly (to me) the --forked flag is a very wide brush which slows down the entire testsuite dramatically when there may only be a few test requiring boxedness. See #23 too.

RonnyPfannschmidt commented 4 years ago

mypersonal oppinion is that this is better served on top of actual xdist, forked has a number of drawbacks, however xdist is not in a shape to support those details (lack of support for multiprocessing among other things)

untitaker commented 4 years ago

@RonnyPfannschmidt I don't understand what you mean. Why do you both think that xdist would serve #23 better and that it can't serve #23 at the same time?

RonnyPfannschmidt commented 4 years ago

@untitaker foked actually forks the interpreter, kills the stdio connection and then at a random later point sends all reports via the stdio fds of the child to the parent

xdist on the other hand starts a interpreter, collects tests completely separate, then just runs that test

so using xdist machinery would make for better isolation and better execution/reporting

however the scheduling mechanisms and the process execution mechanisms in xdist need a feature extension for this use-case to work nicely

untitaker commented 4 years ago

Does xdist run each test in a separate process, or does it fork to create a workerpool? I definetly need one process per boxed test.

From my understanding you have collection per-worker in xdist but I don't see how that's beneficial to me here.

Right now I cannot use parallelization of xdist for my own purposes anyway because I have certain tests that cannot be run in parallel (because they write to a shared pg)

RonnyPfannschmidt commented 4 years ago

what you describe is the current behaviour of xist, what i described is an additional scheduling mechanism that would allow to collect exactly single test for a specific sub-process so that specific tests could be deferred to processes that exit after running a single test