running pytest with slipcover doesn't load plugins (was "doesn't load pytest.ini")

[rlskoeser]

Running slipcover with pytest as described in the readme doesn't work for me; I think this is because it isn't picking up my pytest.ini file.

Running python -m pytest works as expected; running python -m slipcover -m pytest errors — I think based on the error I'm getting it's because of the missing configuration from my pytest.ini. (I'm trying it out with a django project, which requires DJANGO_SETTINGS_MODULE to be defined when not running tests through django.)

When I ran on a project without a pytest.ini file it worked fine.

[jaltmayerpizzorno]

Hi, thank you. Can you provide a more concrete example? I ran a quick try by creating a pytest.ini with a syntax error and got the syntax error both with running pytest and python3 -m slipcover -m pytest.

[rlskoeser]

I was trying to find a simpler case than the project I was first testing with (started there because it's the one I'm actively working on) — and discovered it doesn't error on that project, which does have a pytest.ini file!

In case it's useful, here's the PR where I'm trying out slipcover on that project https://github.com/Princeton-CDH/parasolr/pull/72 — all the tests are passing (although I don't think code coverage is actually being reported, still need to figure out that piece).

I will investigate further on the django project where I first encountered the problem and see what I can figure out.

[rlskoeser]

I think that maybe the problem is that it's not loading my pytest plugins... Not sure how best to investigate or construct a minimal test case, though

[jaltmayerpizzorno]

That would agree with issues reported by @okken. I'll give this a try ASAP.

[okken]

Yep. Sounds the same. For reproduction: Any project works to reproduce with pytest-xdist.

• have a test file to run
• pip install pytest-xdist
• pytest -n 2
• That should run on two cores, but the important part is the -n flag is defined by the plugin
• python -m slipcover -m pytest
• That will error with no -n param defined

[jaltmayerpizzorno]

Thank you, Brian.

@rlskoeser What pytest plugin(s) you are using?

[jaltmayerpizzorno]

I can reproduce the issue and have been debugging it. It looks like none of the plugins are being found.

[jaltmayerpizzorno]

The plugin loading is now fixed... the way slipcover manipulated sys.meta_path was throwing off the importlib mechanism that pytest relies upon for finding plugins.

I'm verifying results, benchmarking it, etc. and will make a new release once ready.

@okken, I haven't looked into that yet, but I don't think pytest-xdist will work correctly yet. Slipcover currently gathers the coverage information in memory; multithreaded tests are fine, but anything using multiprocessing will require some way to coordinate across the processes (e.g., writing to files and assembling the results back from them).

[jaltmayerpizzorno]

@rlskoeser could you try again with the build from pip install -i https://test.pypi.org/simple/ slipcover==0.1.3.dev202205030931 ? Thank you!

@okken, you can try as well, but I still expect to have to add some kind of support for pytest-xdist.

[jaltmayerpizzorno]

I haven't heard back from @rlskoeser, but confirmed that plugins are now being load. The fix will be in v0.1.4.

[rlskoeser]

Sorry for the delay — just tested with the dev release you asked me to try and it works!

[jaltmayerpizzorno]

Awesome! And no worries.