Console stuck in 'Connecting to kernel' when connecting to a venv which has matplotlib installed without pyqt5.

mscheltienne commented 3 years ago

Problem Description

I have a python installation 3.8.10 (also tried 3.9.6) on Windows. On this installation, I have spyder installed via pip. I have multiple venv in a folder: Documents\pyvenv. Each venv has the spyder-kernels installed so that the global spyder installation can have a working IPyton console connected to the proper kernel by changing the setting:

Settings > Python interpreter > Use the following python interpreter

My antivirus/Firewall has both my python installation and my venv whitelisted. Today, some of the IPython console could connect to some of the venv and not to others, staying stuck on 'Connecting to kernel'.

It took me a while to figure out what was causing this: having matplotlib installed on the venv without pyqt5. I was not expecting that.. and I can't figure out the link between matplotlib, pyqt5 and the spyder-kernels.

What steps reproduce the problem?

Create a venv: py -m venv test and activate it.
Update default package: (test) py -m pip install --upgrade pip setuptools wheel
Install spyder-kernels: (test) py -m pip install spyder-kernels

At this point, everything is working.

Install matplotlib: (test) py -m pip install matplotlib

We can wait a while...

Install pyqt5: (test) py -m pip install pyqt5

And it's back !

Notes

I tested this for a while, in multiple venv and on 2 installations of python. It doesn't matter if pyqt5 is installed before or after matplotlib; but as long as both are not installed in the venv, the console is stuck in 'Connecting to kernel'.

Versions

Spyder version: 5.0.5
Spyder-kernels version: 2.0.5
Python version: 3.8.10 / 3.9.6
QtPy version: 1.9.0
PyQt version: 5.12.3
Operating System name/version: Windows 10

Dependencies

# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  1.6.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0               :  7.25.0 (OK)
jedi =0.17.2                  :  0.17.2 (OK)
jsonschema >=3.2.0            :  3.2.0 (OK)
keyring >=17.0.0              :  23.0.1 (OK)
nbconvert >=4.0               :  6.1.0 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
paramiko >=2.4.0              :  2.7.2 (OK)
parso =0.7.0                  :  0.7.0 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.8.0 (OK)
pygments >=2.0                :  2.9.0 (OK)
pylint >=1.0                  :  2.9.3 (OK)
pyls >=0.36.2;<1.0.0          :  0.36.2 (OK)
pyls_black >=0.4.6            :  0.4.7 (OK)
pyls_spyder >=0.3.2;<0.4.0    :  0.3.2 (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.2.0 (OK)
qtawesome >=1.0.2             :  1.0.3 (OK)
qtconsole >=5.1.0             :  5.1.1 (OK)
qtpy >=1.5.0                  :  1.9.0 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=39.0.0           :  57.2.0 (OK)
sphinx >=0.6.6                :  4.1.1 (OK)
spyder_kernels >=2.0.4;<2.1.0 :  2.0.5 (OK)
textdistance >=4.2.0          :  4.2.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.3 (OK)
zmq >=17                      :  22.1.0 (OK)

# Optional:
cython >=0.21                 :  None (NOK)
matplotlib >=2.0.0            :  None (NOK)
numpy >=1.7                   :  None (NOK)
pandas >=1.1.1                :  None (NOK)
scipy >=0.17.0                :  None (NOK)
sympy >=0.7.3                 :  None (NOK)

mscheltienne commented 3 years ago

Now that I think about it, I do have the Qt5 backend set in the settings of the IPython console.

andfoy commented 3 years ago

Hi @mscheltienne, thanks for the detailed issue report. @ccordoba12, do you know what could be happening here? Also, are you using one of our installers?

mscheltienne commented 3 years ago

@andfoy Hello, no this is not through one of the installers but through pip.

A few guess/important points:

matplotlib is not installed in the parent/global install of python (as shown in the optional dependencies of spyder)
In the venv; while matplotlib is not installed, the kernel/console doesn't need a backend to plot and thus works.
In the venv, once matplotlib is installed, and as the Pyqt5 backend is selected in the settings for the IPython console, the kernel/console needs a backend and thus doesn't work/initialize if the backend dependencies are not installed in the venv.

When the spyder-kernels is not installed, a very nice message asking to install it in the venv is displayed in the console. A similar message checking the required dependencies (if matplotlib is installed, it also needs the backend) and asking to install them would be helpful. I don't know much about the kernel and the IPYthon console, maybe there are also over dependencies that are not checked?

I guess this problem was not observed often, as for most people, the kernel is in the same environment as spyder; which does require pyqt5 to be installed in the first place.

ccordoba12 commented 3 years ago

Now that I think about it, I do have the Qt5 backend set in the settings of the IPython console.

This is really the problem. We're not doing a check of the selected Matplotlib backend in the kernel to determine if we can actually set it or not. It shouldn't be easier to add it though and show a message to users telling that it's missing.

However, we perform checks for the Automatic backend, i.e. we check if Qt5, then Tk can be set. If not, we resort to the Inline backend. So I'd suggest you to use that option in the meantime.

spyder-ide / spyder