[Bug]: `macosx` backend is default in `matplotlib>3.9.0`, breaks plots in VS Code

[Simon-Stone]

Bug summary

I am using a Jupyter notebook in VS Code Version: 1.93.1. on MacOS Ventura 13.4.1 and Python 3.12.5

With matplotlib<=3.9.0, I can plot visualizations inline within VS Code. The reported backend is by default inline and everything is fine.

Starting with version 3.9.1, the default backend apparently switched to macosx. When I try to plot something now, a new Python window pops up, but no plot is created unless I add plt.show().

Is this the intended behavior? It breaks essentially all of my notebooks because I would have to go in and add %matplotlib inline to make things work again.

Code for reproduction

import matplotlib.pyplot as plt

import matplotlib

print(matplotlib.get_backend())

plt.plot([0, 1, 2], [1, 2, 3])

Actual outcome

With matplotlib<=3.9.0:

With matplotlib>3.9.0:

Expected outcome

Default backend should be inline in matplotlib>3.9.0.

Additional information

No response

Operating system

MacOS Ventura 13.4.1

Matplotlib Version

3.9.0, 3.9.1, 3.9.2

Matplotlib Backend

inline, macosx

Python version

3.12.5

Jupyter version

6.5.1

Installation

pip

[tacaswell]

@ianthomas23 This is probably more fallout from moving the backend mappings around?

[ianthomas23]

This is not intended, the default matplotib backend in JupyterLab and Notebook should still be the inline backend.

I cannot reproduce this using a new virtual environment on Sonoma 14.6.1 with Python 3.12.6, Matplotlib 3.9.2 and VSCode 1.93.1. Here's my output:

I note that for matplotlib.get_backend() you get the short name inline whereas I get the fully qualified module://matplotlib_inline.backend_inline although the former is shorthand for the latter so may not be significant. But it is a clue.

I think you are using an unusually old versions of the jupyter packages compared to latest matplotlib. ipykernel and ipython are probably the key ones. This is what I have in my new env:

$ jupyter --version
Selected Jupyter core packages...
IPython          : 8.27.0
ipykernel        : 6.29.5
ipywidgets       : 8.1.5
jupyter_client   : 8.6.3
jupyter_core     : 5.7.2
jupyter_server   : 2.14.2
jupyterlab       : 4.2.5
nbclient         : 0.10.0
nbconvert        : 7.16.4
nbformat         : 5.10.4
notebook         : 7.2.2
qtconsole        : not installed
traitlets        : 5.14.3

If by a Jupyter version of 6.5.1 you mean the version of notebook then that is from October 2022 and if you have a similar age of ipykernel and ipython I wouldn't be surprised at some strange results.

Can you do two things?

1. Run jupyter --version and post your output here, then we can see exactly what versions of the important packages you are using.

2. Create new virtual environment and install recent packages (matplotlib and notebook may be enough). I think this should work fine for you.

[Simon-Stone]

Thanks for engaging with this.

I created a fresh virtual environment and used it for what I described above. Here is the output of jupyter --version:

IPython          : 8.27.0
ipykernel        : 6.29.5
ipywidgets       : not installed
jupyter_client   : 8.6.3
jupyter_core     : 5.7.2
jupyter_server   : not installed
jupyterlab       : not installed
nbclient         : not installed
nbconvert        : not installed
nbformat         : not installed
notebook         : not installed
qtconsole        : not installed
traitlets        : 5.14.3

I then ran pip install notebook, and now jupyter --version reports:

Selected Jupyter core packages...
IPython          : 8.27.0
ipykernel        : 6.29.5
ipywidgets       : not installed
jupyter_client   : 8.6.3
jupyter_core     : 5.7.2
jupyter_server   : 2.14.2
jupyterlab       : 4.2.5
nbclient         : 0.10.0
nbconvert        : 7.16.4
nbformat         : 5.10.4
notebook         : 7.2.2
qtconsole        : not installed
traitlets        : 5.14.3

The error is still exactly the same, with the same reported backends.

[ianthomas23]

OK, let's try removing VSCode from the workflow. Can you use jupyter lab or jupyter notebook to run the test code in a browser?

[Simon-Stone]

Good thinking. Here is what I did:

python3 -m venv .venv
source .venv/bin/activate
pip install jupyter ipykernel matplotlib
jupyter notebook

• Notebook browser starts up
• Start test notebook
• Select kernel .venv (Kernel->Change Kernel...)
• Run code

The reported backend is macosx, and the behavior is as described above. In addition, I am getting the following message in the ServerApp log when running the plt.plot command:

[IPKernelApp] WARNING | GUI event loop or pylab initialization failed

Downgrading to 3.9.0 also fixes the behavior, just like from within VS Code.

[ianthomas23]

The warning comes from here: https://github.com/ipython/ipython/blob/68a3854c3e582de0e29dba22f483729c01c6f38b/IPython/core/shellapp.py#L297

The next line shows a traceback, can you copy and paste that from the logs too?

[Simon-Stone]

Small correction: The warning appears every time the kernel is restarted.

Weirdly, there is no traceback. Here is the output from a couple of kernel restarts:

[I 2024-09-23 11:37:59.717 ServerApp] Kernel restarted: 3c53522f-3143-40d8-81ab-9ecab90c6db3 [I 2024-09-23 11:37:59.723 ServerApp] Starting buffering for 3c53522f-3143-40d8-81ab-9ecab90c6db3:23ad432f-7f8e-41ea-828e-98359f647db2 [I 2024-09-23 11:37:59.726 ServerApp] Connecting to kernel 3c53522f-3143-40d8-81ab-9ecab90c6db3. [I 2024-09-23 11:37:59.726 ServerApp] Restoring connection for 3c53522f-3143-40d8-81ab-9ecab90c6db3:23ad432f-7f8e-41ea-828e-98359f647db2 [IPKernelApp] WARNING | GUI event loop or pylab initialization failed [I 2024-09-23 11:38:07.787 ServerApp] Kernel restarted: 3c53522f-3143-40d8-81ab-9ecab90c6db3 [I 2024-09-23 11:38:07.794 ServerApp] Starting buffering for 3c53522f-3143-40d8-81ab-9ecab90c6db3:23ad432f-7f8e-41ea-828e-98359f647db2 [I 2024-09-23 11:38:07.797 ServerApp] Connecting to kernel 3c53522f-3143-40d8-81ab-9ecab90c6db3. [I 2024-09-23 11:38:07.797 ServerApp] Restoring connection for 3c53522f-3143-40d8-81ab-9ecab90c6db3:23ad432f-7f8e-41ea-828e-98359f647db2 [IPKernelApp] WARNING | GUI event loop or pylab initialization failed

[ianthomas23]

I am running out of ideas here.

Matplotlib 3.9.0 is about the right time for things to break with backends as there was a major refactor to move IPython's management of backends to Matplotlib where it more correctly belongs. #27948 was the PR on the Matplotlib side for this, and there were a few glitches fixed later such as #28332 and ipython/ipython#14451.

But as I cannot reproduce the problem it is somewhat difficult to understand it. What it feels like now (and I have been wrong before :smile:) is that you have some overriding of the default Matplotlib backend somewhere such as an environment variable or rcParams, or some preamble code that is run when you start a kernel or notebook which does some backend setting or manipulation. There have been occasions where Jupyter extensions have done something like this (though not to Matplotlib as far as I know) and their forced changes have affected things they shouldn't.

If it was my mac, what I would try next is to create a new user who therefore doesn't have any Matplotlib or Jupyter configuration set up, create a venv for them and re-try the test.

I suppose we should also consider that although we are both using the same package versions, we are using different python installations. I'm using mamba and I guess you are using homebrew? The difference here may be important perhaps in regard to the venvs being fully isolated or not.

[Simon-Stone]

Oh wow, I am so sorry for having you chase this down. I found the culprit:

I remembered that a long time ago I wanted the default figure format to be SVG, so I created a file ~/.ipython/profile_default/ipython_kernel_config.py and put the following in it:

c.InteractiveShellApp.matplotlib = 'inline'
c.InlineBackend.figure_formats = {'svg'}

I removed it just now, and everything works as expected with >3.9.0!

Again, so sorry for this coming back to "user error".

I am curious, though, if you know why this breaks after 3.9.0, and if you had a different suggestion to globally make SVG the default format?