search

dev-petrov / pytest-lazy-fixtures

Allows you to use fixtures in @pytest.mark.parametrize.
MIT License
55 stars 2 forks source link

Version 1.0.7 fails with spyder's tests as a replacement for pytest-lazy-fixture #17

Open juliangilbey opened 4 months ago

juliangilbey commented 4 months ago

Hi! I package Spyder for Debian, and we are migrating to pytest 8.x. Spyder uses pytest-lazy-fixture for a handful of tests, but they have pinned pytest to 6.x. So to address https://github.com/spyder-ide/spyder/issues/21935 I tried using pytest-lazy-fixtures as a drop-in replacement for pytest-lazy-fixture, but it doesn't work (even though according to the documentation it probably should). Here's how to reproduce what I did. I'm working on a Debian testing machine with the relevant libraries installed, and I have pyenv installed. I roughly follow the script in spyder 5.4.4 called .github/scripts/install.sh and did the following (output not shown):

cd /tmp
wget https://github.com/spyder-ide/spyder/archive/refs/tags/v5.5.4.tar.gz
tar zxpf v5.5.4.tar.gz
pyenv shell 3.11.9
python -m venv spyder-311
source spyder-311/bin/activate
cd spyder-5.5.4
python -m pip install -U pip setuptools wheel build
pip install -e '.[test]'
pip uninstall pyqt5 pyqt5-qt5 pyqt5-sip pyqtwebengine pyqtwebengine-qt5 -q -y
pip install pyqt5==5.12.* pyqtwebengine==5.12.*
# upgrade pytest to 7.x
pip install pytest==7.4.4
pushd spyder/app/tests/spyder-boilerplate
pip install --no-deps -q -e .
popd
python -bb -X dev -m build
python -bb -X dev -m pip install --no-deps --force-reinstall dist/spyder*.whl

Now we can run pytest on the one test file which uses pytest-lazy-fixture:

(spyder-311) euler:/tmp/spyder-5.5.4 $ python -m pytest spyder/plugins/completion/providers/languageserver/tests/test_client.py
============================= test session starts ==============================
platform linux -- Python 3.11.9, pytest-7.4.4, pluggy-1.5.0
PyQt5 5.12.3 -- Qt runtime 5.12.10 -- Qt compiled 5.12.10
rootdir: /tmp/spyder-5.5.4
configfile: pytest.ini
plugins: qt-4.4.0, flaky-3.8.1, order-1.2.1, cov-5.0.0, timeout-2.3.1, mock-3.14.0, lazy-fixture-0.6.3
collected 12 items                                                             

spyder/plugins/completion/providers/languageserver/tests/test_client.py . [  8%]
...........                                                              [100%]

=============================== warnings summary ===============================
[... snipped lots of deprecation warnings ...]
======================= 12 passed, 208 warnings in 8.61s =======================

We now edit this file to use pytest-lazy-fixtures instead: here is the diff:

--- spyder/plugins/completion/providers/languageserver/tests/test_client.py.orig2024-05-06 06:28:02.252832014 +0100
+++ spyder/plugins/completion/providers/languageserver/tests/test_client.py 2024-05-06 06:28:22.268593625 +0100
@@ -24,8 +24,8 @@

 @pytest.fixture(scope='module', params=[
-    pytest.lazy_fixture('lsp_provider'),
-    pytest.lazy_fixture('lsp_stdio_provider')])
+    pytest.lazy_fixtures('lsp_provider'),
+    pytest.lazy_fixtures('lsp_stdio_provider')])
 def lsp_client_and_completion(request):
     """Create an LSP client/completion pair."""
     completion = CompletionManager()

and we update the installed packages:

pip uninstall -y pytest-lazy-fixture
pip install pytest-lazy-fixtures==1.0.7

and try again:

(spyder-311) euler:/tmp/spyder-5.5.4 $ python -m pytest spyder/plugins/completion/providers/languageserver/tests/test_client.py
============================= test session starts ==============================
platform linux -- Python 3.11.9, pytest-7.4.4, pluggy-1.5.0
PyQt5 5.12.3 -- Qt runtime 5.12.10 -- Qt compiled 5.12.10
rootdir: /tmp/spyder-5.5.4
configfile: pytest.ini
plugins: qt-4.4.0, flaky-3.8.1, order-1.2.1, lazy-fixtures-1.0.7, cov-5.0.0, timeout-2.3.1, mock-3.14.0
collected 12 items                                                             

spyder/plugins/completion/providers/languageserver/tests/test_client.py . [  8%]
FFFF..FFFF.                                                              [100%]

=================================== FAILURES ===================================
_______________________ test_get_signature[lsp_provider] _______________________

lsp_client_and_completion = (<spyder.plugins.completion.providers.languageserver.client.LSPClient object at 0x7f502b97d630>, <spyder.plugins.completion.providers.languageserver.tests.test_client.CompletionManager object at 0x7f502a569b40>)
qtbot = <pytestqt.qtbot.QtBot object at 0x7f502a5f6190>

    @pytest.mark.order(3)
    def test_get_signature(lsp_client_and_completion, qtbot):
        client, completion = lsp_client_and_completion

        # Parameters to perform a textDocument/didChange request
        params = {
            'file': 'test.py',
            'language': 'python',
            'version': 1,
            'text': "import os\nos.walk(\n",
            'codeeditor': completion,
            'requires_response': False
        }

        # Perform the request
>       with qtbot.waitSignal(completion.sig_response, timeout=30000):
E       pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject) not emitted after 30000 ms

spyder/plugins/completion/providers/languageserver/tests/test_client.py:76: TimeoutError
______________________ test_get_completions[lsp_provider] ______________________
[... rest of failure logs snipped - they're all pretty much the same ...]
=============================== warnings summary ===============================
[... snipped as before ...]
=========================== short test summary info ============================
FAILED spyder/plugins/completion/providers/languageserver/tests/test_client.py::test_get_signature[lsp_provider] - pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject...
FAILED spyder/plugins/completion/providers/languageserver/tests/test_client.py::test_get_completions[lsp_provider] - pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject...
FAILED spyder/plugins/completion/providers/languageserver/tests/test_client.py::test_go_to_definition[lsp_provider] - pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject...
FAILED spyder/plugins/completion/providers/languageserver/tests/test_client.py::test_local_signature[lsp_provider] - pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject...
FAILED spyder/plugins/completion/providers/languageserver/tests/test_client.py::test_get_signature[lsp_stdio_provider] - pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject...
FAILED spyder/plugins/completion/providers/languageserver/tests/test_client.py::test_get_completions[lsp_stdio_provider] - pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject...
FAILED spyder/plugins/completion/providers/languageserver/tests/test_client.py::test_go_to_definition[lsp_stdio_provider] - pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject...
FAILED spyder/plugins/completion/providers/languageserver/tests/test_client.py::test_local_signature[lsp_stdio_provider] - pytestqt.exceptions.TimeoutError: Signal sig_response(QString,PyQt_PyObject...
============ 8 failed, 4 passed, 208 warnings in 243.97s (0:04:03) =============

Any ideas why pytest-lazy-fixtures might be failing in this situation?