spyder-ide / spyder

Official repository for Spyder - The Scientific Python Development Environment
https://www.spyder-ide.org
MIT License
8.34k stars 1.62k forks source link

Failing tests with Spyder 5.3.x and Debian unstable #17651

Closed juliangilbey closed 11 months ago

juliangilbey commented 2 years ago

Issue Report Checklist

Problem Description

Thanks all for your great work on getting 5.3.0 released! I'm pleased to say that all of the tests that were failing with 5.2.x now work. There are just a new problematic tests, and I don't know enough to understand what is wrong. But besides these, it looks very good, so I'll upload 5.3.0 to Debian unstable (with these tests disabled) and then submit a further issue if any of them fail on architectures other than amd64.

test_mainwindow.py segfaults

spyder/app/tests/test_mainwindow.py regularly segfaults. Here's an example of running this test alone. (My runtests.py is a slightly modified version of the official version, introducing the --homedir option.) I don't know whether there's enough information here to help identify the source of the crash, though.

euler:~/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1 $ CI=true HOME=/tmp/homedir xvfb-run -a -s "-screen 0 1024x768x24 +extension GLX" python3.9 runtests.py --homedir=/tmp/homedir --run-slow -v 'spyder/app/tests/test_mainwindow.py'
Pytest Arguments: ['-vv', '-rw', '--durations=10', '-W ignore::UserWarning', '--run-slow', '-v', 'spyder/app/tests/test_mainwindow.py']
============================= test session starts ==============================
platform linux -- Python 3.9.12, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /usr/bin/python3.9
cachedir: .pytest_cache
PySide2 5.15.2 -- Qt runtime 5.15.2 -- Qt compiled 5.15.2
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/.hypothesis/examples')
rootdir: /home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1, configfile: pytest.ini
plugins: lazy-fixture-0.6.3, xdist-2.5.0, anyio-3.5.0, order-1.0.1, mock-3.6.1, forked-1.4.0, cov-3.0.0, timeout-2.1.0, qt-4.0.2, dependency-0.5.1, flaky-3.7.0, hypothesis-6.36.0
collected 108 items                                                            

spyder/app/tests/test_mainwindow.py::test_single_instance_and_edit_magic Fatal Python error: Segmentation fault

Thread 0x00007f623f7fe640 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/threading.py", line 574 in wait
  File "/usr/lib/python3/dist-packages/IPython/core/history.py", line 829 in run
  File "/usr/lib/python3/dist-packages/IPython/core/history.py", line 60 in only_when_enabled
  File "<decorator-gen-17>", line 2 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007f626d7fa640 (most recent call first):
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/onlinehelp/pydoc_patch.py", line 938 in serve_until_quit
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/onlinehelp/pydoc_patch.py", line 967 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Current thread 0x00007f6472393740 (most recent call first):
  File "/usr/lib/python3/dist-packages/qtconsole/console_widget.py", line 342 in __init__
  File "/usr/lib/python3/dist-packages/qtconsole/history_console_widget.py", line 27 in __init__
  File "/usr/lib/python3/dist-packages/qtconsole/frontend_widget.py", line 165 in __init__
  File "/usr/lib/python3/dist-packages/qtconsole/jupyter_widget.py", line 118 in __init__
  File "/usr/lib/python3/dist-packages/qtconsole/rich_jupyter_widget.py", line 55 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/debugging.py", line 75 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/debugging.py", line 195 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/shell.py", line 119 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/client.py", line 146 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/main_widget.py", line 1548 in create_new_client
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/plugin.py", line 521 in create_new_client
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/plugin.py", line 389 in on_mainwindow_visible
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/mainwindow.py", line 1169 in post_visible_setup
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/utils.py", line 300 in create_window
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/mainwindow.py", line 1989 in main
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/start.py", line 234 in main
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/tests/test_mainwindow.py", line 331 in main_window
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 925 in call_fixture_func
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 1126 in pytest_fixture_setup
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 1072 in execute
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 687 in _compute_fixture_value
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 601 in _get_active_fixturedef
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 581 in getfixturevalue
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 568 in _fillfixtures
  File "/usr/lib/python3/dist-packages/pytest_lazyfixture.py", line 39 in fill
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 1647 in setup
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 449 in prepare
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 150 in pytest_runtest_setup
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 255 in <lambda>
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 311 in from_call
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 254 in call_runtest_hook
  File "/usr/lib/python3/dist-packages/flaky/flaky_pytest_plugin.py", line 138 in call_and_report
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 120 in runtestprotocol
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
  File "/usr/lib/python3/dist-packages/flaky/flaky_pytest_plugin.py", line 94 in pytest_runtest_protocol
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 348 in pytest_runtestloop
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 323 in _main
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 269 in wrap_session
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 162 in main
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/runtests.py", line 48 in run_pytest
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/runtests.py", line 78 in main
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/runtests.py", line 82 in <module>
Segmentation fault

test_get_hints fails

spyder/plugins/editor/widgets/tests/test_hints_and_calltips.py::test_get_hints fails with all three parameters, both with Python 3.9 and Python 3.10; here's one of the test logs:

___________________________ test_get_hints[params0] ____________________________

qtbot = <pytestqt.qtbot.QtBot object at 0x7f10c7c49a00>
completions_codeeditor = (<spyder.plugins.editor.widgets.codeeditor.CodeEditor object at 0x7f10a29a89d0>, <spyder.plugins.completion.plugin.CompletionPlugin object at 0x7f10a2cc5820>)
params = ('"".format', '-> str')
capsys = <_pytest.capture.CaptureFixture object at 0x7f10a2ef6fd0>

    @pytest.mark.slow
    @pytest.mark.order(2)
    @pytest.mark.skipif(sys.platform == 'darwin', reason='Fails on Mac')
    @pytest.mark.parametrize('params', [
            # Parameter, Expected Output
            ('"".format', '-> str'),
            ('import math', 'module'),
            (TEST_TEXT, TEST_DOCSTRING)
        ]
    )
    def test_get_hints(qtbot, completions_codeeditor, params, capsys):
        """Test that the editor is returning hover hints."""
        code_editor, _ = completions_codeeditor
        param, expected_output_text = params

        # Move mouse to another position to be sure the hover is displayed when
        # the cursor is put on top of the tested word.
        qtbot.mouseMove(code_editor, QPoint(400, 400))

        # Set text in editor
        code_editor.set_text(param)

        # Get cursor coordinates
        code_editor.moveCursor(QTextCursor.End)
        qtbot.keyPress(code_editor, Qt.Key_Left)

        # Wait a bit in case the window manager repositions the window.
        qtbot.wait(1000)

        # Position cursor on top of word we want the hover for.
        x, y = code_editor.get_coordinates('cursor')
        point = code_editor.calculate_real_position(QPoint(x, y))

        # Get hover and compare
        with qtbot.waitSignal(code_editor.sig_display_object_info,
                              timeout=30000) as blocker:
            qtbot.mouseMove(code_editor, point)
            qtbot.mouseClick(code_editor, Qt.LeftButton, pos=point)
>           qtbot.waitUntil(lambda: code_editor.tooltip_widget.isVisible(),
                            timeout=10000)
E           pytestqt.exceptions.TimeoutError: waitUntil timed out in 10000 milliseconds

spyder/plugins/editor/widgets/tests/test_hints_and_calltips.py:171: TimeoutError

I do seem to get the expected behaviour when I perform these actions manually.

Versions

Dependencies

# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.0.2 (OK)
IPython >=7.31.1              :  8.0.1 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.0 (OK)
jellyfish >=0.7               :  0.8.9 (OK)
jsonschema >=3.2.0            :  3.2.0 (OK)
keyring >=17.0.0              :  23.5.0 (OK)
nbconvert >=4.0               :  6.4.4 (OK)
numpydoc >=0.6.0              :  1.2.1 (OK)
parso >=0.7.0;<0.9.0          :  0.8.1 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.0 (OK)
pygments >=2.0                :  2.11.2 (OK)
pylint >=2.5.0                :  2.12.2 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.4.1;<1.5.0          :  1.4.1 (OK)
pylsp_black >=1.2.0           :  1.2.0 (OK)
qdarkstyle >=3.0.2;<3.1.0     :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.1.1 (OK)
qtconsole >=5.3.0;<5.4.0      :  5.3.0 (OK)
qtpy >=2.0.1                  :  2.0.1 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  59.6.0 (OK)
sphinx >=0.6.6                :  4.3.2 (OK)
spyder_kernels >=2.3.0;<2.4.0 :  2.3.0 (OK)
textdistance >=4.2.0          :  4.2.2 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.7 (OK)
xdg >=0.26                    :  0.27 (OK)
zmq >=17                      :  22.3.0 (OK)

# Optional:
cython >=0.21                 :  0.29.28 (OK)
matplotlib >=3.0.0            :  3.5.1 (OK)
numpy >=1.7                   :  1.21.5 (OK)
pandas >=1.1.1                :  1.3.5 (OK)
scipy >=0.17.0                :  1.7.3 (OK)
sympy >=0.7.3                 :  1.9 (OK)
juliangilbey commented 2 years ago

Having now tried uploading Spyder 5.3.0 to Debian, two more tests failed. Again, these seem to be somewhat flaky, as they both passed for me. On amd64, we had the following error: https://ci.debian.net/data/autopkgtest/unstable/amd64/s/spyder/20762635/log.gz

[...]
==================================== ERRORS ====================================
_________________ ERROR at setup of test_pdb_eventloop[inline] _________________

qtbot = <pytestqt.qtbot.QtBot object at 0x7fde729b9250>
request = <SubRequest 'ipyconsole' for <Function test_pdb_eventloop[inline]>>
tmpdir = local('/tmp/pytest-of-debci/pytest-0/test_pdb_eventloop_inline_2')

    @pytest.fixture
    def ipyconsole(qtbot, request, tmpdir):
        """IPython console fixture."""
        configuration = ConfigurationManager(conf_path=str(tmpdir))

        class MainWindowMock(QMainWindow):
            def get_spyder_pythonpath(self):
                return configuration.get('main', 'spyder_pythonpath', [])

            def __getattr__(self, attr):
                if attr == 'consoles_menu_actions':
                    return []
                elif attr == 'editor':
                    return None
                else:
                    return Mock()

        # Tests assume inline backend
        configuration.set('ipython_console', 'pylab/backend', 0)

        # Start in a new working directory the console
        use_startup_wdir = request.node.get_closest_marker('use_startup_wdir')
        if use_startup_wdir:
            new_wdir = osp.join(os.getcwd(), NEW_DIR)
            if not osp.exists(new_wdir):
                os.mkdir(new_wdir)
            configuration.set('workingdir', 'console/use_fixed_directory', True)
            configuration.set('workingdir', 'console/fixed_directory', new_wdir)
        else:
            configuration.set('workingdir', 'console/use_fixed_directory', False)
            configuration.set(
                'workingdir', 'console/fixed_directory', get_home_dir())

        # Test the console with a non-ascii temp dir
        non_ascii_dir = request.node.get_closest_marker('non_ascii_dir')
        if non_ascii_dir:
            test_dir = NON_ASCII_DIR
        else:
            test_dir = ''

        # Instruct the console to not use a stderr file
        no_stderr_file = request.node.get_closest_marker('no_stderr_file')
        if no_stderr_file:
            test_no_stderr = 'True'
        else:
            test_no_stderr = ''

        # Use the automatic backend if requested
        auto_backend = request.node.get_closest_marker('auto_backend')
        if auto_backend:
            configuration.set('ipython_console', 'pylab/backend', 1)

        # Use the Tkinter backend if requested
        tk_backend = request.node.get_closest_marker('tk_backend')
        if tk_backend:
            configuration.set('ipython_console', 'pylab/backend', 3)

        # Start a Pylab client if requested
        pylab_client = request.node.get_closest_marker('pylab_client')
        is_pylab = True if pylab_client else False

        # Start a Sympy client if requested
        sympy_client = request.node.get_closest_marker('sympy_client')
        is_sympy = True if sympy_client else False

        # Start a Cython client if requested
        cython_client = request.node.get_closest_marker('cython_client')
        is_cython = True if cython_client else False

        # Use an external interpreter if requested
        external_interpreter = request.node.get_closest_marker(
            'external_interpreter')
        if external_interpreter:
            configuration.set('main_interpreter', 'default', False)
            configuration.set('main_interpreter', 'executable', sys.executable)
        else:
            configuration.set('main_interpreter', 'default', True)
            configuration.set('main_interpreter', 'executable', '')

        # Use the test environment interpreter if requested
        test_environment_interpreter = request.node.get_closest_marker(
            'test_environment_interpreter')
        if test_environment_interpreter:
            configuration.set('main_interpreter', 'default', False)
            configuration.set(
                'main_interpreter', 'executable', get_conda_test_env())
        else:
            configuration.set('main_interpreter', 'default', True)
            configuration.set('main_interpreter', 'executable', '')

        # Conf css_path in the Appeareance plugin
        configuration.set('appearance', 'css_path', CSS_PATH)

        # Create the console and a new client and set environment
        os.environ['IPYCONSOLE_TESTING'] = 'True'
        os.environ['IPYCONSOLE_TEST_DIR'] = test_dir
        os.environ['IPYCONSOLE_TEST_NO_STDERR'] = test_no_stderr
        window = MainWindowMock()
        console = IPythonConsole(parent=window, configuration=configuration)
        console._register()
        console.create_new_client(is_pylab=is_pylab,
                                  is_sympy=is_sympy,
                                  is_cython=is_cython)
        window.setCentralWidget(console.get_widget())

        # Set exclamation mark to True
        configuration.set('ipython_console', 'pdb_use_exclamation_mark', True)

        if os.name == 'nt':
            qtbot.addWidget(window)

        with qtbot.waitExposed(window):
            window.resize(640, 480)
            window.show()

        # Wait until the window is fully up
        qtbot.waitUntil(lambda: console.get_current_shellwidget() is not None)
        shell = console.get_current_shellwidget()
        try:
>           qtbot.waitUntil(lambda: shell._prompt_html is not None,
                            timeout=SHELL_TIMEOUT)
E                           pytestqt.exceptions.TimeoutError: waitUntil timed out in 20000 milliseconds

/tmp/autopkgtest-lxc.zg_qnxgt/downtmp/build.YcW/src/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py:219: TimeoutError
---------------------------- Captured stdout setup -----------------------------
Python 3.9.12 (main, Mar 24 2022, 13:02:21) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

info_page
<!--
    kernel_error.html
    ~~~~~~~~~~~~~~~~~

    Kernel error template for the IPython Console

    :copyright: Copyright by the Spyder Project Contributors
    :license: MIT license
-->

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html>

<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
    <link rel="stylesheet" href="file:////tmp/autopkgtest-lxc.zg_qnxgt/downtmp/build.YcW/src/spyder/plugins/help/utils/static/css/default.css" type="text/css"/>
</head>

<body>
  <div class="panel panel-danger">
    <div class="panel-heading">
      <div class="panel-title">An error ocurred while starting the kernel</div>
    </div>
    <div class="panel-body">
      <tt>[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled<br></tt>
    </div>
  </div>
</body>

</html>

---------------------------- Captured stdout setup -----------------------------
Python 3.9.12 (main, Mar 24 2022, 13:02:21) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

info_page
<!--
    kernel_error.html
    ~~~~~~~~~~~~~~~~~

    Kernel error template for the IPython Console

    :copyright: Copyright by the Spyder Project Contributors
    :license: MIT license
-->

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html>

<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
    <link rel="stylesheet" href="file:////tmp/autopkgtest-lxc.zg_qnxgt/downtmp/build.YcW/src/spyder/plugins/help/utils/static/css/default.css" type="text/css"/>
</head>

<body>
  <div class="panel panel-danger">
    <div class="panel-heading">
      <div class="panel-title">An error ocurred while starting the kernel</div>
    </div>
    <div class="panel-body">
      <tt>[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled<br></tt>
    </div>
  </div>
</body>

</html>

---------------------------- Captured stdout setup -----------------------------

info_page
<!--
    kernel_error.html
    ~~~~~~~~~~~~~~~~~

    Kernel error template for the IPython Console

    :copyright: Copyright by the Spyder Project Contributors
    :license: MIT license
-->

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html>

<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
    <link rel="stylesheet" href="file:////tmp/autopkgtest-lxc.zg_qnxgt/downtmp/build.YcW/src/spyder/plugins/help/utils/static/css/default.css" type="text/css"/>
</head>r 

<body>
  <div class="panel panel-danger">
    <div class="panel-heading">
      <div class="panel-title">An error ocurred while starting the kernel</div>
    </div>
    <div class="panel-body">
      <tt>Exception in thread Heartbeat:<br>Traceback (most recent call last):<br>  File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner<br>    self.run()<br>  File "/usr/lib/python3/dist&#8209packages/ipykernel/heartbeat.py", line 97, in run<br>    self._bind_socket()<br>  File "/usr/lib/python3/dist&#8209packages/ipykernel/heartbeat.py", line 77, in _bind_socket<br>    self._try_bind_socket()<br>  File "fo/usr/lib/python3/dist&#8209packages/ipykernel/heartbeat.py", line 64, in _try_bind_socket<br>    return self.socket.bind('%s://%s' % (self.transport, self.ip) + c + str(self.port))<br>  File "/usr/lib/python3/dist&#8209packages/zmq/sugar/socket.py", line 214, in bind<br>    super().bind(addr)<br>  File "zmq/backend/cython/socket.pyx", line 540, in zmq.backend.cython.socket.Socket.bind<br>  File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc<br>zmq.error.ZMQError: Address already in use<br></tt>
    </div>
  </div>
</body>

</html>

(As in a previous report, debugpy is not yet packaged for Debian, and is proving to be quite challenging. But that is not the cause of this failure.)

On arm64, a different test had a different error, which is very similar:

==================================== ERRORS ====================================
________________________ ERROR at setup of test_pdb_out ________________________

qtbot = <pytestqt.qtbot.QtBot object at 0xfffe6b97c310>
request = <SubRequest 'ipyconsole' for <Function test_pdb_out>>
tmpdir = local('/tmp/pytest-of-debci/pytest-0/test_pdb_out0')

    @pytest.fixture
    def ipyconsole(qtbot, request, tmpdir):
        """IPython console fixture."""
        configuration = ConfigurationManager(conf_path=str(tmpdir))

        class MainWindowMock(QMainWindow):
            def get_spyder_pythonpath(self):
                return configuration.get('main', 'spyder_pythonpath', [])

            def __getattr__(self, attr):
                if attr == 'consoles_menu_actions':
                    return []
                elif attr == 'editor':
                    return None
                else:
                    return Mock()

        # Tests assume inline backend
        configuration.set('ipython_console', 'pylab/backend', 0)

        # Start in a new working directory the console
        use_startup_wdir = request.node.get_closest_marker('use_startup_wdir')
        if use_startup_wdir:
            new_wdir = osp.join(os.getcwd(), NEW_DIR)
            if not osp.exists(new_wdir):
                os.mkdir(new_wdir)
            configuration.set('workingdir', 'console/use_fixed_directory', True)
            configuration.set('workingdir', 'console/fixed_directory', new_wdir)
        else:
            configuration.set('workingdir', 'console/use_fixed_directory', False)
            configuration.set(
                'workingdir', 'console/fixed_directory', get_home_dir())

        # Test the console with a non-ascii temp dir
        non_ascii_dir = request.node.get_closest_marker('non_ascii_dir')
        if non_ascii_dir:
            test_dir = NON_ASCII_DIR
        else:
            test_dir = ''

        # Instruct the console to not use a stderr file
        no_stderr_file = request.node.get_closest_marker('no_stderr_file')
        if no_stderr_file:
            test_no_stderr = 'True'
        else:
            test_no_stderr = ''

        # Use the automatic backend if requested
        auto_backend = request.node.get_closest_marker('auto_backend')
        if auto_backend:
            configuration.set('ipython_console', 'pylab/backend', 1)

        # Use the Tkinter backend if requested
        tk_backend = request.node.get_closest_marker('tk_backend')
        if tk_backend:
            configuration.set('ipython_console', 'pylab/backend', 3)

        # Start a Pylab client if requested
        pylab_client = request.node.get_closest_marker('pylab_client')
        is_pylab = True if pylab_client else False

        # Start a Sympy client if requested
        sympy_client = request.node.get_closest_marker('sympy_client')
        is_sympy = True if sympy_client else False

        # Start a Cython client if requested
        cython_client = request.node.get_closest_marker('cython_client')
        is_cython = True if cython_client else False

        # Use an external interpreter if requested
        external_interpreter = request.node.get_closest_marker(
            'external_interpreter')
        if external_interpreter:
            configuration.set('main_interpreter', 'default', False)
            configuration.set('main_interpreter', 'executable', sys.executable)
        else:
            configuration.set('main_interpreter', 'default', True)
            configuration.set('main_interpreter', 'executable', '')

        # Use the test environment interpreter if requested
        test_environment_interpreter = request.node.get_closest_marker(
            'test_environment_interpreter')
        if test_environment_interpreter:
            configuration.set('main_interpreter', 'default', False)
            configuration.set(
                'main_interpreter', 'executable', get_conda_test_env())
        else:
            configuration.set('main_interpreter', 'default', True)
            configuration.set('main_interpreter', 'executable', '')

        # Conf css_path in the Appeareance plugin
        configuration.set('appearance', 'css_path', CSS_PATH)

        # Create the console and a new client and set environment
        os.environ['IPYCONSOLE_TESTING'] = 'True'
        os.environ['IPYCONSOLE_TEST_DIR'] = test_dir
        os.environ['IPYCONSOLE_TEST_NO_STDERR'] = test_no_stderr
        window = MainWindowMock()
        console = IPythonConsole(parent=window, configuration=configuration)
        console._register()
        console.create_new_client(is_pylab=is_pylab,
                                  is_sympy=is_sympy,
                                  is_cython=is_cython)
        window.setCentralWidget(console.get_widget())

        # Set exclamation mark to True
        configuration.set('ipython_console', 'pdb_use_exclamation_mark', True)

        if os.name == 'nt':
            qtbot.addWidget(window)

        with qtbot.waitExposed(window):
            window.resize(640, 480)
            window.show()

        # Wait until the window is fully up
        qtbot.waitUntil(lambda: console.get_current_shellwidget() is not None)
        shell = console.get_current_shellwidget()
        try:
>           qtbot.waitUntil(lambda: shell._prompt_html is not None,
                            timeout=SHELL_TIMEOUT)
E                           pytestqt.exceptions.TimeoutError: waitUntil timed out in 20000 milliseconds

/tmp/autopkgtest-lxc.ymsgfp3u/downtmp/build.g0t/src/spyder/plugins/ipythonconsole/tests/test_ipythonconsole.py:219: TimeoutError
---------------------------- Captured stdout setup -----------------------------
Python 3.9.12 (main, Mar 24 2022, 13:02:21) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

info_page
<!--
    kernel_error.html
    ~~~~~~~~~~~~~~~~~

    Kernel error template for the IPython Console

    :copyright: Copyright by the Spyder Project Contributors
    :license: MIT license
-->

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html>

<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
    <link rel="stylesheet" href="file:////tmp/autopkgtest-lxc.ymsgfp3u/downtmp/build.g0t/src/spyder/plugins/help/utils/static/css/default.css" type="text/css"/>
</head>

<body>
  <div class="panel panel-danger">
    <div class="panel-heading">
      <div class="panel-title">An error ocurred while starting the kernel</div>
    </div>
    <div class="panel-body">
      <tt>[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled<br></tt>
    </div>
  </div>
</body>

</html>

If you have any clue why we might be getting these errors, or whether just marking them as flaky would be good enough, I'd very much appreciate it!

ccordoba12 commented 2 years ago

Hey @juliangilbey, thanks for reporting. About the failures you found:

test_mainwindow.py segfaults

That depends on the main window being properly closed, as far as I know, but @dalthviz could know more. In any case, I don't know how we could fix it because it's passing fine here on Github.

test_get_hints fails

That depends on the place the cursor is positioned on the screen, which could give problems on headless systems. I wouldn't worry about it and simply skip it because it's not possible to check where the cursor is in CIs.

Again, these seem to be somewhat flaky, as they both passed for me

It seems your CIs are taking too much time to start an IPython console, hence the failure. Not our fault really but of your CIs.

dalthviz commented 2 years ago

Thanks @juliangilbey for the feedback! Regarding the test_mainwindow.py segfaults that's something I'm checking as part of #17580 at #17622 but at least here on GitHub that problem could be being workarounded since for each job multiple attempts are contemplated (in the GitHub actions workflow we achieve this like this:

https://github.com/spyder-ide/spyder/blob/9f292bcd2d6b461dd79c04021fb11111f5644e24/.github/workflows/test-linux.yml#L127

Also, maybe something you can help us with this is running those tests with gdb in your setup (maybe with that we will be able to get some more info of the cause of the segfaults). For that you can run something like this:

https://github.com/spyder-ide/spyder/blob/9f292bcd2d6b461dd79c04021fb11111f5644e24/.github/workflows/test-linux.yml#L123

Regarding test_get_hints, I would say that marking it as flaky with something like @flaky(max_runs=3) could work :+1: At https://github.com/spyder-ide/spyder/pull/17622 I did that with a test from the code analysis/pylint plugin

Regarding test_pdb_eventloop[inline], I would say that marking as flaky could help since the error shows something related to ZMQ and the heartbeat channel socket unable to be binded to an address which could be caused due to the kernel taking to much time to start as @ccordoba12 suggested.

Regarding test_pdb_out seems like is more clearly related to debugpy since the error shown indicates that debugging will not be enabled due to something related to debugpy not being defined (maybe @impact27 knows better about this kind of warning message):

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled
juliangilbey commented 2 years ago

Hi @ccordoba12, thanks for your thoughts on this!

Hey @juliangilbey, thanks for reporting. About the failures you found: [...]

test_get_hints fails

That depends on the place the cursor is positioned on the screen, which could give problems on headless systems. I wouldn't worry about it and simply skip it because it's not possible to check where the cursor is in CIs.

OK, will skip it then.

Again, these seem to be somewhat flaky, as they both passed for me

It seems your CIs are taking too much time to start an IPython console, hence the failure. Not our fault really but of your CIs.

Ah, interesting. So I'll increase the timeouts and see if that helps.

juliangilbey commented 2 years ago

Hi @dalthviz, thanks! Lots of things to think about here!

Thanks @juliangilbey for the feedback! Regarding the test_mainwindow.py segfaults that's something I'm checking as part of #17580 at #17622 but at least here on GitHub that problem could be being workarounded since for each job multiple attempts are contemplated (in the GitHub actions workflow we achieve this like this:

https://github.com/spyder-ide/spyder/blob/9f292bcd2d6b461dd79c04021fb11111f5644e24/.github/workflows/test-linux.yml#L127

Yes, I saw that; I didn't particularly want to do this if I could help it, though!

Also, maybe something you can help us with this is running those tests with gdb in your setup (maybe with that we will be able to get some more info of the cause of the segfaults). For that you can run something like this:

https://github.com/spyder-ide/spyder/blob/9f292bcd2d6b461dd79c04021fb11111f5644e24/.github/workflows/test-linux.yml#L123

Here's the output of running this:

euler:~/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1 $ CI=TRUE HOME=/tmp/homedir xvfb-run --auto-servernum gdb -return-child-result -batch -ex r -ex py-bt --args python3.10 runtests.py --homedir=/tmp/homedir --run-slow 'spyder/app/tests/test_mainwindow.py'
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Pytest Arguments: ['-vv', '-rw', '--durations=10', '-W ignore::UserWarning', '--run-slow', 'spyder/app/tests/test_mainwindow.py']
============================= test session starts ==============================
platform linux -- Python 3.10.4, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /usr/bin/python3.10
cachedir: .pytest_cache
PySide2 5.15.2 -- Qt runtime 5.15.2 -- Qt compiled 5.15.2
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/.hypothesis/examples')
rootdir: /home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1, configfile: pytest.ini
plugins: lazy-fixture-0.6.3, xdist-2.5.0, anyio-3.5.0, order-1.0.1, mock-3.6.1, forked-1.4.0, cov-3.0.0, timeout-2.1.0, qt-4.0.2, dependency-0.5.1, flaky-3.7.0, hypothesis-6.36.0
collecting ... [New Thread 0x7fffd1ec9640 (LWP 425952)]
[New Thread 0x7fffd16c8640 (LWP 425953)]
[New Thread 0x7fffc0ec7640 (LWP 425954)]
[New Thread 0x7fffc06c6640 (LWP 425955)]
[New Thread 0x7fffafec5640 (LWP 425956)]
[New Thread 0x7fffaf6c4640 (LWP 425957)]
[New Thread 0x7fff9eec3640 (LWP 425958)]
[New Thread 0x7fff9e6c2640 (LWP 425959)]
[New Thread 0x7fff8dec1640 (LWP 425960)]
[New Thread 0x7fff856c0640 (LWP 425961)]
[New Thread 0x7fff84ebf640 (LWP 425962)]
[New Thread 0x7fff746be640 (LWP 425963)]
[New Thread 0x7fff6bebd640 (LWP 425964)]
[New Thread 0x7fff636bc640 (LWP 425965)]
[New Thread 0x7fff5aebb640 (LWP 425966)]
[New Thread 0x7fff5a6ba640 (LWP 425967)]
[New Thread 0x7fff49eb9640 (LWP 425968)]
[New Thread 0x7fff416b8640 (LWP 425969)]
[New Thread 0x7fff38eb7640 (LWP 425970)]
[New Thread 0x7fff386b6640 (LWP 425971)]
[New Thread 0x7fff27eb5640 (LWP 425972)]
[New Thread 0x7fff1f6b4640 (LWP 425973)]
[New Thread 0x7fff1eeb3640 (LWP 425974)]
[New Thread 0x7fff166b2640 (LWP 425975)]
[New Thread 0x7fff05eb1640 (LWP 425976)]
[New Thread 0x7ffefd6b0640 (LWP 425977)]
[New Thread 0x7ffef4eaf640 (LWP 425978)]
[New Thread 0x7ffeec6ae640 (LWP 425979)]
[New Thread 0x7ffee3ead640 (LWP 425980)]
[New Thread 0x7ffedb6ac640 (LWP 425981)]
[New Thread 0x7ffed2eab640 (LWP 425982)]
[Detaching after vfork from child process 425993]
[Detaching after vfork from child process 425994]
collected 108 items                                                            

spyder/app/tests/test_mainwindow.py::test_single_instance_and_edit_magic [New Thread 0x7ffec6e94640 (LWP 426111)]
[New Thread 0x7ffec5586640 (LWP 426140)]
[New Thread 0x7ffec4d85640 (LWP 426141)]
[New Thread 0x7ffeb6e94640 (LWP 426142)]
[New Thread 0x7ffeb6693640 (LWP 426143)]
[New Thread 0x7ffeb5e92640 (LWP 426144)]
[New Thread 0x7ffeb5691640 (LWP 426145)]
[New Thread 0x7ffeb4e90640 (LWP 426146)]
[New Thread 0x7ffe97fff640 (LWP 426147)]
[New Thread 0x7ffe977fe640 (LWP 426148)]
[New Thread 0x7ffe96ffd640 (LWP 426149)]
[New Thread 0x7ffe967fc640 (LWP 426151)]
[New Thread 0x7ffe95ffb640 (LWP 426152)]
[New Thread 0x7ffe957fa640 (LWP 426153)]
[New Thread 0x7ffe94ff9640 (LWP 426154)]
[New Thread 0x7ffe77fff640 (LWP 426155)]
[New Thread 0x7ffe6ffff640 (LWP 426156)]
[New Thread 0x7ffe777fe640 (LWP 426157)]
[New Thread 0x7ffe76ffd640 (LWP 426158)]
[New Thread 0x7ffe767fc640 (LWP 426159)]
[New Thread 0x7ffe75ffb640 (LWP 426160)]
[New Thread 0x7ffe757fa640 (LWP 426161)]
[New Thread 0x7ffe74ff9640 (LWP 426162)]
[New Thread 0x7ffe6f7fe640 (LWP 426163)]
[New Thread 0x7ffe6effd640 (LWP 426164)]
[New Thread 0x7ffe6e7fc640 (LWP 426165)]
[New Thread 0x7ffe6dffb640 (LWP 426166)]
[New Thread 0x7ffe6d7fa640 (LWP 426167)]
[New Thread 0x7ffe6cff9640 (LWP 426168)]
[New Thread 0x7ffe37fff640 (LWP 426169)]
[New Thread 0x7ffe2ffff640 (LWP 426170)]
[New Thread 0x7ffe377fe640 (LWP 426171)]
[New Thread 0x7ffe36ffd640 (LWP 426172)]
[New Thread 0x7ffe367fc640 (LWP 426176)]
[New Thread 0x7ffe35ffb640 (LWP 426182)]
[New Thread 0x7ffe357fa640 (LWP 426183)]
[New Thread 0x7ffe34ff9640 (LWP 426187)]
[Detaching after vfork from child process 426190]
[New Thread 0x7ffe2ecfe640 (LWP 426236)]
[New Thread 0x7ffe2dfc0640 (LWP 426237)]
[Detaching after fork from child process 426238]
[Thread 0x7ffed2eab640 (LWP 425982) exited]
[Thread 0x7ffedb6ac640 (LWP 425981) exited]
[Thread 0x7ffee3ead640 (LWP 425980) exited]
[Thread 0x7ffeec6ae640 (LWP 425979) exited]
[Thread 0x7ffef4eaf640 (LWP 425978) exited]
[Thread 0x7ffefd6b0640 (LWP 425977) exited]
[Thread 0x7fff05eb1640 (LWP 425976) exited]
[Thread 0x7fff166b2640 (LWP 425975) exited]
[Thread 0x7fff1eeb3640 (LWP 425974) exited]
[Thread 0x7fff1f6b4640 (LWP 425973) exited]
[Thread 0x7fff27eb5640 (LWP 425972) exited]
[Thread 0x7fff386b6640 (LWP 425971) exited]
[Thread 0x7fff38eb7640 (LWP 425970) exited]
[Thread 0x7fff416b8640 (LWP 425969) exited]
[Thread 0x7fff49eb9640 (LWP 425968) exited]
[Thread 0x7fff5a6ba640 (LWP 425967) exited]
[Thread 0x7fff5aebb640 (LWP 425966) exited]
[Thread 0x7fff636bc640 (LWP 425965) exited]
[Thread 0x7fff6bebd640 (LWP 425964) exited]
[Thread 0x7fff746be640 (LWP 425963) exited]
[Thread 0x7fff84ebf640 (LWP 425962) exited]
[Thread 0x7fff856c0640 (LWP 425961) exited]
[Thread 0x7fff8dec1640 (LWP 425960) exited]
[Thread 0x7fff9e6c2640 (LWP 425959) exited]
[Thread 0x7fff9eec3640 (LWP 425958) exited]
[Thread 0x7fffaf6c4640 (LWP 425957) exited]
[Thread 0x7fffafec5640 (LWP 425956) exited]
[Thread 0x7fffc06c6640 (LWP 425955) exited]
[Thread 0x7fffc0ec7640 (LWP 425954) exited]
[Thread 0x7fffd16c8640 (LWP 425953) exited]
[Thread 0x7fffd1ec9640 (LWP 425952) exited]
[Detaching after fork from child process 426239]
[Detaching after fork from child process 426240]
[New Thread 0x7ffed2eab640 (LWP 426245)]
[New Thread 0x7ffedb6ac640 (LWP 426246)]
[New Thread 0x7ffee3ead640 (LWP 426247)]
[New Thread 0x7ffeec6ae640 (LWP 426248)]
[New Thread 0x7fffd1ec9640 (LWP 426249)]
[New Thread 0x7fffd16c8640 (LWP 426250)]
[New Thread 0x7fffc0ec7640 (LWP 426251)]
[New Thread 0x7fffc06c6640 (LWP 426252)]
[New Thread 0x7fffafec5640 (LWP 426253)]
[New Thread 0x7fffaf6c4640 (LWP 426254)]
[New Thread 0x7fff9eec3640 (LWP 426255)]
[New Thread 0x7fff9e6c2640 (LWP 426256)]
[New Thread 0x7fff856c0640 (LWP 426257)]
[New Thread 0x7fff84ebf640 (LWP 426258)]
[New Thread 0x7fff746be640 (LWP 426302)]
[Detaching after vfork from child process 426303]
[New Thread 0x7fff6bebd640 (LWP 426318)]
[New Thread 0x7fff636bc640 (LWP 426319)]
[New Thread 0x7fff5aebb640 (LWP 426320)]
[New Thread 0x7fff5a6ba640 (LWP 426321)]
[Thread 0x7fff5aebb640 (LWP 426320) exited]
[Thread 0x7fff746be640 (LWP 426302) exited]
[Detaching after vfork from child process 426349]
[Detaching after vfork from child process 426353]
[New Thread 0x7fff746be640 (LWP 426354)]
[New Thread 0x7fff5aebb640 (LWP 426355)]
[Detaching after vfork from child process 426359]
[New Thread 0x7fff416b8640 (LWP 426360)]
[New Thread 0x7fff38eb7640 (LWP 426361)]
[New Thread 0x7fff386b6640 (LWP 426380)]

Thread 1 "python3.10" received signal SIGSEGV, Segmentation fault.
0x00007ffff740b17d in ?? () from /usr/lib/python3/dist-packages/PyQt5/sip.cpython-310-x86_64-linux-gnu.so
Traceback (most recent call first):
  File "/usr/lib/python3/dist-packages/qtconsole/console_widget.py", line 342, in __init__
    self.increase_font_size = QtWidgets.QAction("Bigger Font",
  File "/usr/lib/python3/dist-packages/qtconsole/history_console_widget.py", line 27, in __init__
    super().__init__(*args, **kw)
  File "/usr/lib/python3/dist-packages/qtconsole/frontend_widget.py", line 165, in __init__
    super().__init__(*args, **kw)
  File "/usr/lib/python3/dist-packages/qtconsole/jupyter_widget.py", line 118, in __init__
    super().__init__(*args, **kw)
  File "/usr/lib/python3/dist-packages/qtconsole/rich_jupyter_widget.py", line 55, in __init__
    super().__init__(*args, **kw)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/debugging.py", line 75, in __init__
    super(DebuggingHistoryWidget, self).__init__(*args, **kwargs)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/debugging.py", line 195, in __init__
    super(DebuggingWidget, self).__init__(*args, **kwargs)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/shell.py", line 119, in __init__
    super(ShellWidget, self).__init__(*args, **kw)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/client.py", line 146, in __init__
    self.shellwidget = ShellWidget(
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/main_widget.py", line 1548, in create_new_client
    client = ClientWidget(self, id_=client_id,
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/plugin.py", line 521, in create_new_client
    self.get_widget().create_new_client(
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/plugin.py", line 389, in on_mainwindow_visible
    self.create_new_client(give_focus=False)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/mainwindow.py", line 1169, in post_visible_setup
    plugin.on_mainwindow_visible()
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/utils.py", line 300, in create_window
    main.post_visible_setup()
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/mainwindow.py", line 1989, in main
    window = create_window(MainWindow, app, None, options, None)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/start.py", line 234, in main
    return mainwindow.main(options, args)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/tests/test_mainwindow.py", line 331, in main_window
    window = start.main()
  <built-in method next of module object at remote 0x7ffff7b3c5e0>
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 925, in call_fixture_func
    fixture_result = next(generator)
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 1126, in pytest_fixture_setup
    result = call_fixture_func(fixturefunc, request, kwargs)
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 1072, in execute
    result = hook.pytest_fixture_setup(fixturedef=self, request=request)
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 687, in _compute_fixture_value
    fixturedef.execute(request=subrequest)
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 601, in _get_active_fixturedef
    self._compute_fixture_value(fixturedef)
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 581, in getfixturevalue
    fixturedef = self._get_active_fixturedef(argname)
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 568, in _fillfixtures
    item.funcargs[argname] = self.getfixturevalue(argname)
  File "/usr/lib/python3/dist-packages/pytest_lazyfixture.py", line 39, in fill
    _fillfixtures()
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 1647, in setup
    self._request._fillfixtures()
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 449, in prepare
    col.setup()
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 150, in pytest_runtest_setup
    item.session._setupstate.prepare(item)
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 255, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 311, in from_call
    result: Optional[TResult] = func()
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 254, in call_runtest_hook
    return CallInfo.from_call(
  File "/usr/lib/python3/dist-packages/flaky/flaky_pytest_plugin.py", line 138, in call_and_report
    call = call_runtest_hook(item, when, **kwds)
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 120, in runtestprotocol
    rep = call_and_report(item, "setup", log)
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 109, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "/usr/lib/python3/dist-packages/flaky/flaky_pytest_plugin.py", line 94, in pytest_runtest_protocol
    self.runner.pytest_runtest_protocol(item, nextitem)
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 348, in pytest_runtestloop
    item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 323, in _main
    config.hook.pytest_runtestloop(session=session)
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 269, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/runtests.py", line 48, in run_pytest
    errno = pytest.main(pytest_args)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/runtests.py", line 78, in main
    run_pytest(run_slow=test_args.run_slow, extra_args=pytest_args)
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/runtests.py", line 82, in <module>
    main()

I hope there's something useful here.

Regarding test_get_hints, I would say that marking it as flaky with something like @flaky(max_runs=3) could work +1 At #17622 I did that with a test from the code analysis/pylint plugin

OK, I tried that, but with no success. So I'll just skip it.

Regarding test_pdb_eventloop[inline], I would say that marking as flaky could help since the error shows something related to ZMQ and the heartbeat channel socket unable to be binded to an address which could be caused due to the kernel taking to much time to start as @ccordoba12 suggested.

OK; I'm instead extending the timeouts everywhere, as that seems a more robust approach.

Regarding test_pdb_out seems like is more clearly related to debugpy since the error shown indicates that debugging will not be enabled due to something related to debugpy not being defined (maybe @impact27 knows better about this kind of warning message):

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

I don't think so, as it runs fine on my machine and the error message was about timeouts again. I'm extending the timeout; we'll see if that helps.

Best wishes, Julian

dalthviz commented 2 years ago

Checking the traceback from gdb seems that the segfault is happening over QtConsole (at initialization of some actions) when creating a new IPython Console when Spyder is starting:

File "/usr/lib/python3/dist-packages/qtconsole/console_widget.py", line 342, in __init__
    self.increase_font_size = QtWidgets.QAction("Bigger Font",

Not totally sure why but maybe trying to run the test suit from QtConsole could help us further debug or maybe even checking if launching a standalone instance of QtConsole shows some error or segfaults could help.

juliangilbey commented 2 years ago

Hi @dalthviz, I can launch a QtConsole successfully. But it still crashes when running the test suite. Here's the kernel display:

Jupyter QtConsole 5.3.0
Python 3.9.12 (main, Mar 24 2022, 13:02:21) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %cd ~/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/
/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1

In [2]: import runtests

In [3]: runtests.run_pytest(run_slow=True, extra_args=["spyder/app/tests/test_mainwindow.py"])
Pytest Arguments: ['-vv', '-rw', '--durations=10', '-W ignore::UserWarning', '--run-slow', 'spyder/app/tests/test_mainwindow.py']
============================= test session starts ==============================
platform linux -- Python 3.9.12, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
PySide2 5.15.2 -- Qt runtime 5.15.2 -- Qt compiled 5.15.2
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/.hypothesis/examples')
rootdir: /home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1, configfile: pytest.ini
plugins: typeguard-2.11.1, lazy-fixture-0.6.3, xdist-2.5.0, anyio-3.5.0, order-1.0.1, mock-3.6.1, forked-1.4.0, cov-3.0.0, timeout-2.1.0, qt-4.0.2, dependency-0.5.1, flaky-3.7.0, hypothesis-6.36.0
collecting ... collected 108 items

spyder/app/tests/test_mainwindow.py::test_single_instance_and_edit_magic SKIPPED [  0%]
spyder/app/tests/test_mainwindow.py::test_default_plugin_actions SKIPPED [  1%]
spyder/app/tests/test_mainwindow.py::test_change_types_in_varexp 
Kernel died, restarting

and here's the terminal output:

euler:/tmp $ jupyter-qtconsole 
Fatal Python error: Segmentation fault

Thread 0x00007efd72072640 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/threading.py", line 574 in wait
  File "/usr/lib/python3/dist-packages/IPython/core/history.py", line 829 in run
  File "/usr/lib/python3/dist-packages/IPython/core/history.py", line 58 in needs_sqlite
  File "<decorator-gen-17>", line 2 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007efd73fff640 (most recent call first):
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/onlinehelp/pydoc_patch.py", line 938 in serve_until_quit
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/onlinehelp/pydoc_patch.py", line 967 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007eff16ffd640 (most recent call first):
  File "/usr/lib/python3/dist-packages/zmq/utils/garbage.py", line 49 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007eff177fe640 (most recent call first):
  File "/usr/lib/python3/dist-packages/ipykernel/parentpoller.py", line 36 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007eff17fff640 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/threading.py", line 574 in wait
  File "/usr/lib/python3/dist-packages/IPython/core/history.py", line 829 in run
  File "/usr/lib/python3/dist-packages/IPython/core/history.py", line 58 in needs_sqlite
  File "<decorator-gen-17>", line 2 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007eff2cff9640 (most recent call first):
  File "/usr/lib/python3.9/selectors.py", line 469 in select
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1869 in _run_once
  File "/usr/lib/python3.9/asyncio/base_events.py", line 601 in run_forever
  File "/usr/lib/python3/dist-packages/tornado/platform/asyncio.py", line 199 in start
  File "/usr/lib/python3/dist-packages/ipykernel/control.py", line 22 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007eff2d7fa640 (most recent call first):
  File "/usr/lib/python3/dist-packages/ipykernel/iostream.py", line 326 in _watch_pipe_fd
  File "/usr/lib/python3.9/threading.py", line 910 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007eff2dffb640 (most recent call first):
  File "/usr/lib/python3/dist-packages/ipykernel/iostream.py", line 326 in _watch_pipe_fd
  File "/usr/lib/python3.9/threading.py", line 910 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007eff2f7fe640 (most recent call first):
  File "/usr/lib/python3/dist-packages/ipykernel/heartbeat.py", line 104 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x00007eff2ffff640 (most recent call first):
  File "/usr/lib/python3.9/selectors.py", line 469 in select
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1869 in _run_once
  File "/usr/lib/python3.9/asyncio/base_events.py", line 601 in run_forever
  File "/usr/lib/python3/dist-packages/tornado/platform/asyncio.py", line 199 in start
  File "/usr/lib/python3/dist-packages/ipykernel/iostream.py", line 82 in _thread_main
  File "/usr/lib/python3.9/threading.py", line 910 in run
  File "/usr/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 930 in _bootstrap

Current thread 0x00007eff3865c740 (most recent call first):
  File "/usr/lib/python3/dist-packages/qtconsole/console_widget.py", line 342 in __init__
  File "/usr/lib/python3/dist-packages/qtconsole/history_console_widget.py", line 27 in __init__
  File "/usr/lib/python3/dist-packages/qtconsole/frontend_widget.py", line 165 in __init__
  File "/usr/lib/python3/dist-packages/qtconsole/jupyter_widget.py", line 118 in __init__
  File "/usr/lib/python3/dist-packages/qtconsole/rich_jupyter_widget.py", line 55 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/debugging.py", line 75 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/debugging.py", line 195 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/shell.py", line 119 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/client.py", line 146 in __init__
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/widgets/main_widget.py", line 1548 in create_new_client
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/plugin.py", line 521 in create_new_client
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/plugins/ipythonconsole/plugin.py", line 389 in on_mainwindow_visible
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/mainwindow.py", line 1169 in post_visible_setup
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/utils.py", line 300 in create_window
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/mainwindow.py", line 1989 in main
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/start.py", line 248 in main
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/spyder/app/tests/test_mainwindow.py", line 331 in main_window
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 925 in call_fixture_func
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 1126 in pytest_fixture_setup
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 1072 in execute
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 687 in _compute_fixture_value
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 601 in _get_active_fixturedef
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 581 in getfixturevalue
  File "/usr/lib/python3/dist-packages/_pytest/fixtures.py", line 568 in _fillfixtures
  File "/usr/lib/python3/dist-packages/pytest_lazyfixture.py", line 39 in fill
  File "/usr/lib/python3/dist-packages/_pytest/python.py", line 1647 in setup
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 449 in prepare
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 150 in pytest_runtest_setup
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 255 in <lambda>
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 311 in from_call
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 254 in call_runtest_hook
  File "/usr/lib/python3/dist-packages/flaky/flaky_pytest_plugin.py", line 138 in call_and_report
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 120 in runtestprotocol
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
  File "/usr/lib/python3/dist-packages/flaky/flaky_pytest_plugin.py", line 94 in pytest_runtest_protocol
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 348 in pytest_runtestloop
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 323 in _main
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 269 in wrap_session
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 39 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 265 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 162 in main
  File "/home/jdg/debian/spyder-packages/spyder/build-area/spyder-5.3.0+dfsg1/runtests.py", line 48 in run_pytest
  File "/tmp/ipykernel_651340/1320161637.py", line 1 in <module>
  File "/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py", line 3457 in run_code
  File "/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py", line 3377 in run_ast_nodes
  File "/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py", line 3185 in run_cell_async
  File "/usr/lib/python3/dist-packages/IPython/core/async_helpers.py", line 78 in _pseudo_sync_runner
  File "/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py", line 2960 in _run_cell
  File "/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py", line 2914 in run_cell
  File "/usr/lib/python3/dist-packages/ipykernel/zmqshell.py", line 532 in run_cell
  File "/usr/lib/python3/dist-packages/ipykernel/ipkernel.py", line 355 in do_execute
  File "/usr/lib/python3/dist-packages/ipykernel/kernelbase.py", line 664 in execute_request
  File "/usr/lib/python3/dist-packages/ipykernel/kernelbase.py", line 369 in dispatch_shell
  File "/usr/lib/python3/dist-packages/ipykernel/kernelbase.py", line 462 in process_one
  File "/usr/lib/python3/dist-packages/ipykernel/kernelbase.py", line 473 in dispatch_queue
  File "/usr/lib/python3.9/asyncio/events.py", line 80 in _run
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1905 in _run_once
  File "/usr/lib/python3.9/asyncio/base_events.py", line 601 in run_forever
  File "/usr/lib/python3/dist-packages/tornado/platform/asyncio.py", line 199 in start
  File "/usr/lib/python3/dist-packages/ipykernel/kernelapp.py", line 677 in start
  File "/usr/lib/python3/dist-packages/traitlets/config/application.py", line 846 in launch_instance
  File "/usr/lib/python3/dist-packages/ipykernel_launcher.py", line 16 in <module>
  File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main
[JupyterQtConsoleApp] KernelRestarter: restarting kernel (1/5), keep random ports
[JupyterQtConsoleApp] WARNING | kernel restarted

I get pretty much identical output if I run a plain runtests.run_pytest(run_slow=True).

dalthviz commented 2 years ago

Oh sorry @juliangilbey I think I didn't explain things correctly 😅 I meant to say that maybe running the QtConsole test suit in your setup could help us see if there is something over QtConsole that needs to be fixed (since the Spyder test suit is failing when initializing the console widget which is code located at QtConsole). However, with the test you did, I think is clear that something inside the test suit is somehow faulty when reaching the QtConsole code. Maybe something related with pytest-qt or the creation of the QtApplication that is causing the segfault when the QtConsole code starts creating some actions 🤔

juliangilbey commented 2 years ago

Hello, and sorry for the slow reply on this one. Here's an update on this.

Best wishes, Julian

juliangilbey commented 2 years ago

Hi there! Here's an update with Spyder 5.3.1 on amd64.

Things are generally better than with 5.3.0! Only two tests are still consistently failing; the first is spyder/app/tests/test_mainwindow.py::test_run_cython_code

Here is an example of it (and we still don't have debugpy, but that seems to be unrelated):

_____________________________ test_run_cython_code _____________________________

main_window = <spyder.app.mainwindow.MainWindow object at 0x7fd754ada700>
qtbot = <pytestqt.qtbot.QtBot object at 0x7fd754d37fd0>

    @pytest.mark.slow
    @flaky(max_runs=3)
    @pytest.mark.skipif(
        (os.name == 'nt' or sys.platform == 'darwin' or
         parse_version(ipy_release.version) == parse_version('7.11.0')),
        reason="Hard to test on Windows and macOS and fails for IPython 7.11.0")
    def test_run_cython_code(main_window, qtbot):
        """Test all the different ways we have to run Cython code"""
        # ---- Setup ----
        # Get a reference to the code editor widget
        code_editor = main_window.editor.get_focus_widget()

        # ---- Run pyx file ----
        # Load test file
        main_window.editor.load(osp.join(LOCATION, 'pyx_script.pyx'))

        # Run file
        qtbot.keyClick(code_editor, Qt.Key_F5)

        # Get a reference to the namespace browser widget
        nsb = main_window.variableexplorer.current_widget()

        # Wait until an object appears
>       qtbot.waitUntil(lambda: nsb.editor.source_model.rowCount() == 1,
                        timeout=COMPILE_AND_EVAL_TIMEOUT)
E       pytestqt.exceptions.TimeoutError: waitUntil timed out in 30000 milliseconds

/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/test_mainwindow.py:1349: TimeoutError
----------------------------- Captured Qt messages -----------------------------
QtWarningMsg: Attribute Qt::AA_UseSoftwareOpenGL must be set before QCoreApplication is created.
QtWarningMsg: Scenegraph already initialized, setBackend() request ignored
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests')

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests')

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests')

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests')

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests')

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests')

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests')

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.4qx5x6ye/downtmp/build.Ycv/src/spyder/app/tests')

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [2]: 

And the second test almost always fails: spyder/app/tests/test_mainwindow.py::test_ordering_lsp_requests_at_startup

Here is some sample output:

____________________ test_ordering_lsp_requests_at_startup _____________________

main_window = <spyder.app.mainwindow.MainWindow object at 0x7ff9d82ec280>
qtbot = <pytestqt.qtbot.QtBot object at 0x7ff948040250>

    @pytest.mark.slow
    @flaky(max_runs=3)
    @pytest.mark.use_introspection
    @pytest.mark.order(after="test_debug_unsaved_function")
    @pytest.mark.preload_project
    @pytest.mark.skipif(os.name == 'nt', reason='Times out on Windows')
    def test_ordering_lsp_requests_at_startup(main_window, qtbot):
        """
        Test the ordering of requests we send to the LSP at startup when a
        project was left open during the previous session.

        This is a regression test for spyder-ide/spyder#13351.
        """
        # Wait until the LSP server is up.
        code_editor = main_window.editor.get_current_editor()
        qtbot.waitSignal(code_editor.completions_response_signal, timeout=30000)

        # Wait until the initial requests are sent to the server.
        lsp = main_window.completions.get_provider('lsp')
        python_client = lsp.clients['python']
        qtbot.wait(5000)

        expected_requests = [
            'initialize',
            'initialized',
            'workspace/didChangeConfiguration',
            'workspace/didChangeWorkspaceFolders',
            'textDocument/didOpen',
        ]

        skip_intermediate = {
            'initialized': {'workspace/didChangeConfiguration'}
        }

        lsp_requests = python_client['instance']._requests
        start_idx = lsp_requests.index((0, 'initialize'))

        request_order = []
        expected_iter = iter(expected_requests)
        current_expected = next(expected_iter)
        for i in range(start_idx, len(lsp_requests)):
            if current_expected is None:
                break

            _, req_type = lsp_requests[i]
            if req_type == current_expected:
                request_order.append(req_type)
                current_expected = next(expected_iter, None)
            else:
                skip_set = skip_intermediate.get(current_expected, set({}))
                if req_type in skip_set:
                    continue
                else:
>                   assert req_type == current_expected
E                   AssertionError: assert 'textDocument/didChange' == 'textDocument/didOpen'
E                     - textDocument/didOpen
E                     ?                 ^^ -
E                     + textDocument/didChange
E                     ?                 ^^^^^

/tmp/autopkgtest-lxc.0c0pxmav/downtmp/build.gSC/src/spyder/app/tests/test_mainwindow.py:4098: AssertionError
----------------------------- Captured Qt messages -----------------------------
QtWarningMsg: Attribute Qt::AA_UseSoftwareOpenGL must be set before QCoreApplication is created.
QtWarningMsg: Scenegraph already initialized, setBackend() request ignored
--------------------------- Captured stdout teardown ---------------------------
Python 3.9.12 (main, May 14 2022, 05:57:40) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [1]: 
--------------------------- Captured stdout teardown ---------------------------
Python 3.9.12 (main, May 14 2022, 05:57:40) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [1]: 
--------------------------- Captured stdout teardown ---------------------------
Python 3.9.12 (main, May 14 2022, 05:57:40) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [1]: 
--------------------------- Captured stdout teardown ---------------------------
Python 3.9.12 (main, May 14 2022, 05:57:40) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [1]: 
--------------------------- Captured stdout teardown ---------------------------
Python 3.9.12 (main, May 14 2022, 05:57:40) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [1]: 
--------------------------- Captured stdout teardown ---------------------------
Python 3.9.12 (main, May 14 2022, 05:57:40) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [1]: 
--------------------------- Captured stdout teardown ---------------------------
Python 3.9.12 (main, May 14 2022, 05:57:40) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [1]: 
--------------------------- Captured stdout teardown ---------------------------
Python 3.9.12 (main, May 14 2022, 05:57:40) 
Type "copyright", "credits" or "license" for more information.

IPython 7.31.1 -- An enhanced Interactive Python.

[SpyderKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled

In [1]: 

I'm sometimes getting segfaults in test_mainwindow.py, but as ever, these are really hard to track down. I'm also sometimes getting timeouts in our testbed system (there's a time limit of just under 3 hours). Other than that, things seem to be doing fine.

On i386, I'm still excluding test_mainwindow.py (as it regularly segfaults) and test_ipythonconsole.py (likewise), and also spyder/plugins/editor/widgets/tests/test_hints_and_calltips.py::test_get_hints as it repeatedly fails.

Best wishes, Julian

juliangilbey commented 2 years ago

Ah, I may have spoken in too much haste: spyder/plugins/editor/widgets/tests/test_hints_and_calltips.py::test_get_hints is still usually failing, even on amd64. I'll leave it deselected.

dalthviz commented 2 years ago

Thanks for the feedback @juliangilbey ! Just in case, we are currently working fixing some things related to the tests here: https://github.com/spyder-ide/spyder/pull/18106 and also we have the idea to add pytest-timeout to prevent test stalling the test suite here: https://github.com/spyder-ide/spyder/pull/17990

juliangilbey commented 2 years ago

Hi @dalthviz, that sounds great, thanks!

juliangilbey commented 2 years ago

Hi all, here's an update with 5.3.3 on amd64.

_____________________________ test_run_cython_code _____________________________

main_window = <spyder.app.mainwindow.MainWindow object at 0x7fdcb819dea0>
qtbot = <pytestqt.qtbot.QtBot object at 0x7fdc786aa050>

    @pytest.mark.slow
    @flaky(max_runs=3)
    @pytest.mark.skipif(
        (os.name == 'nt' or sys.platform == 'darwin' or
         parse_version(ipy_release.version) == parse_version('7.11.0')),
        reason="Hard to test on Windows and macOS and fails for IPython 7.11.0")
    def test_run_cython_code(main_window, qtbot):
        """Test all the different ways we have to run Cython code"""
        # Wait until the window is fully up
        shell = main_window.ipyconsole.get_current_shellwidget()
        qtbot.waitUntil(
            lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)

        # ---- Setup ----
        # Get a reference to the code editor widget
        code_editor = main_window.editor.get_focus_widget()

        # ---- Run pyx file ----
        # Load test file
        main_window.editor.load(osp.join(LOCATION, 'pyx_script.pyx'))

        # Run file
        qtbot.keyClick(code_editor, Qt.Key_F5)

        # Get a reference to the namespace browser widget
        nsb = main_window.variableexplorer.current_widget()

        # Wait until an object appears
>       qtbot.waitUntil(lambda: nsb.editor.source_model.rowCount() == 1,
                        timeout=COMPILE_AND_EVAL_TIMEOUT)
E       pytestqt.exceptions.TimeoutError: waitUntil timed out in 30000 milliseconds

/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/test_mainwindow.py:1410: TimeoutError
----------------------------- Captured Qt messages -----------------------------
QtWarningMsg: Attribute Qt::AA_UseSoftwareOpenGL must be set before QCoreApplication is created.
QtWarningMsg: Scenegraph already initialized, setBackend() request ignored
---------------------------- Captured stdout setup -----------------------------
Attribute Qt::AA_UseSoftwareOpenGL must be set before QCoreApplication is created.
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests')

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests')

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests')

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests')

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests')

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests')

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests')

In [2]: 
--------------------------- Captured stdout teardown ---------------------------

In [1]: runfile('/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests/pyx_script.pyx', wdir='/tmp/autopkgtest-lxc.tyd2nkfj/downtmp/build.6sp/src/spyder/app/tests')

In [2]: 

Everything else is passing!

ccordoba12 commented 2 years ago

Perhaps the Cython test is failing because you're using a Gcc version that is not compatible with Cython yet. I mean, from the logs you posted it seems that Cython is failing to compile the code passed to it.

So I'd skip it for now and retry it in a month or so, when a new Cython version is out.

juliangilbey commented 2 years ago

Thanks @ccordoba12! There is definitely something a bit weird about the handling of pyx files, though.

Unreproducible experience: I saved a copy of pyx_script.pyx in /tmp (using a shell), then tried opening it in Spyder 5.3.3 and Spyder crashed. It only happened once. The several next times I tried opening it, it opens without difficulty. I can press F5 and the value of a is exactly as expected. So it's not a gcc version issue by the looks of things.

I've tried to follow the source code in spyder to understand what runfile does, and it seems to assume that the script is Python, not Cython. So I'm confused why the stdout at teardown says it is calling runfile at all. Hmm.

Even more, if I save a copy of the pyx_script.pyx file as pyx_script.py (note the .py extension). When I open this in Spyder, it shows a syntax error on the first line, but when I press F5, it runs with no difficulty, correctly calculating a (and the same happens if I called it fake_script.py, so Spyder seems to be analysing the content, not the filename).

On the other hand, if I do the following:

juliangilbey commented 2 years ago

Following on from the above, I've just investigated and opened https://github.com/spyder-ide/spyder/issues/19344

ccordoba12 commented 11 months ago

Closing because I think most of the problem reported here were already addressed.