pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.93k stars 2.65k forks source link

Fixture not found in 8.0 that is found in 7.4.4 #11884

Open larsoner opened 8 months ago

larsoner commented 8 months ago

In this run:

https://github.com/openmeeg/openmeeg/actions/runs/7700673043/job/20985097864?pr=631#step:6:8200

You can see the 3.7 wheel work during testing (which uses 7.4.4), but fail in 3.8 testing (which uses 8.0.0). If I use pytest --fixtures $TEST_PATH the fixture is there:

  ---------------- fixtures defined from openmeeg.tests.conftest ----------------
  data_path [session scope] -- ..\venv-test\lib\site-packages\openmeeg\tests\conftest.py:15
      no docstring available

  run_some_parallel [session scope] -- ..\venv-test\lib\site-packages\openmeeg\tests\conftest.py:27
      Run some stuff in parallel.

But when I run the tests with pytest --tb=short -ra -m 'not slow' -vv $TEST_PATH I get for all tests:

  ============================= test session starts =============================
  platform win32 -- Python 3.8.10, pytest-8.0.0, pluggy-1.4.0 -- C:\Users\runneradmin\AppData\Local\Temp\cibw-run-32upad1x\cp38-win_amd64\venv-test\Scripts\python.exe
  cachedir: .pytest_cache
  rootdir: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-32upad1x\cp38-win_amd64
  collecting ... collected 13 items / 2 deselected / 11 selected

  ..\venv-test\Lib\site-packages\openmeeg\tests\test_doc.py::test_doc <- venv-test\lib\site-packages\openmeeg\tests\test_doc.py ERROR [  9%]

...

  =================================== ERRORS ====================================
  _________________________ ERROR at setup of test_doc __________________________
  file C:\Users\runneradmin\AppData\Local\Temp\cibw-run-32upad1x\cp38-win_amd64\venv-test\lib\site-packages\openmeeg\tests\test_doc.py, line 6
    def test_doc():
  E       fixture 'run_some_parallel' not found
  >       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
  >       use 'pytest --fixtures [testpath]' for help on them.

Notably, this only happens on the Windows cibuildwheel run -- the Linux one in that same GHA is green. So the issue seems isolated to 8.0.0 on Windows. Nonetheless I am a bit mystified at why pytest --fixtures seems to see a different set of fixtures compared to 7.4.4 :shrug:

Not sure how to make this one mimimal but I could put some time into this on my Windows VM at some point if it's not clear enough what's going on.

larsoner commented 8 months ago

(Oh and it shouldn't be a 3.7 vs 3.8 thing because things were fine last week on these same runs!)

bluetech commented 8 months ago

It looks like the conftest is not getting loaded since the data_path also doesn't show up in the "available fixtures". This is probably related to annoying Windows path stuff.

I can probably reconstruct things from above but just to make things easier for me:


Unrelated, but I'm curious: is there a reason you're doing this:

https://github.com/openmeeg/openmeeg/blob/331759d15006fbdf1ff13842baa435ae87c8ef19/wrapping/python/openmeeg/tests/conftest.py#L10

instead of making the fixture autouse=True?

larsoner commented 8 months ago

I can probably reconstruct things from above but just to make things easier for me:

From a new GHA run in a PR sandbox (where I could eventually test a fix branch if you want):

Unrelated, but I'm curious: is there a reason you're doing this:

Last year when I wrote this I didn't understand autouse=True. Thanks for the reminder, I'll remove it!

bluetech commented 7 months ago

@larsoner We've fixed some windows regressions, could you try again with latest main? I suspect it won't fix the issue but maybe it will.

larsoner commented 7 months ago

Looks like on commit b510adf9efcd21e7ff2ab8e93e925f3e8363bb7d just now the failure is still there :disappointed:

bluetech commented 7 months ago

OK, so keeping this on the radar, thanks for checking.