search

pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.96k stars 2.66k forks source link

add_marker with usefixtures no longer works with pytest 8.0.0 #11873

Open rhshadrach opened 8 months ago

rhshadrach commented 8 months ago

I believe this may be related to https://github.com/pytest-dev/pytest/issues/3664.

In pandas, we dynamically add a marker for our doctests so that we don't need import pandas as pd and import numpy as np in every docstring.

https://github.com/pandas-dev/pandas/blob/9008ee5810c09bc907b5fdc36fc3c1dff4a50c55/pandas/conftest.py#L191-L198

The add_doctest_imports fixture is here.

https://github.com/pandas-dev/pandas/blob/9008ee5810c09bc907b5fdc36fc3c1dff4a50c55/pandas/conftest.py#L253-L259

With pytest 8.0.0, add_marker no longer has any effect. Is the intentional, and if so, is there an alternative?

bluetech commented 8 months ago

Very similar to #11759. Would the alternative described there of using an autouse fixture instead of the hook work for pandas?

rhshadrach commented 8 months ago

Thanks! There is a comment in the pandas code that we don't use autouse because of perf. I'll see if I can track down the history there and see if I can reproduce the perf issue.

rhshadrach commented 8 months ago

This was originally changed in pandas in https://github.com/pandas-dev/pandas/pull/45667. It tried undoing that patch locally, seeing negligible differences in runtimes for the test suite. Will see what it looks like on the pandas CI in https://github.com/pandas-dev/pandas/pull/57122.

mroeschke commented 8 months ago

It would be nice if an alternative to an autouse fixture could be used in the future as non-doctests don't need this fixture at all (which is probably +98% of tests in pandas).

The prior motivation for doing this dynamically was me being curious why in the durations output the last-ran unit test TEARDOWN could be one of the longest durations, and removing an autouse=True, doctest_namespace fixture was the culprit.

Zeitsperre commented 2 months ago

@rhshadrach

Sorry to ping here, but I'm trying to understand how your autouse fix in pandas works here. It seems as though no fixtures are called when running $ pytest --doctest-modules src/my_lib. Is there something else going on?

rhshadrach commented 2 months ago

@Zeitsperre

Sorry to ping here...

Pings are welcome!

It seems as though no fixtures are called when running $ pytest --doctest-modules src/my_lib. Is there something else going on?

pandas CI invokes doctests via: python -c "import pandas as pd; pd.test(run_doctests=True)". There may be other ways to invoke them from the command line. However, when running that line, the fixture here below is used.

https://github.com/pandas-dev/pandas/blob/aa4dc71f54764252ff795cc42b1c465e20a204c0/pandas/conftest.py#L245

Zeitsperre commented 2 months ago

@rhshadrach That fixed my issues, thanks!

Still hoping for a better workaround for the autouse=True issue.

Cheers!