`ImportError: cannot import name 'document' from 'js' (unknown location)` unless (patched?) `matplotlib.pyplot` function is called before `matplotlib.rcParams.update`

[posita]

Description

Calling matplotlib.rcParams.update(matplotlib.rcParamsDefault) before any matplotlib.pyplot call will brick the running kernel. More specifically, any call to any matplotlib.pyplot function will raise ImportError: cannot import name 'document' from 'js' (unknown location) thereafter.

Reproduce

Create a new notebook (e.g., in https://jupyterlite.rtfd.io/en/latest/try/lab, but I've been able to reproduce this in several installations going back to 0.1.0b11) and paste the following in the first cell:

import matplotlib.pyplot
# matplotlib.pyplot.clf()  # <-- uncomment this and restarting the kernel will show an empty plot
matplotlib.rcParams.update(matplotlib.rcParamsDefault)  # <-- xor comment this and restarting the kernel will show an empty plot
matplotlib.pyplot.show()

Select Run :arrow_right: Restart Kernel and Run All Cells... (or click the :fast_forward: icon in the tool bar). You'll get a stack trace like the following:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In [1], line 4
      2 # matplotlib.pyplot.clf()  # <-- uncomment this and restarting the kernel will show an empty plot
      3 matplotlib.rcParams.update(matplotlib.rcParamsDefault)  # <-- xor comment this and restarting the kernel will show an empty plot
----> 4 matplotlib.pyplot.show()

File /lib/python3.10/site-packages/pyolite/patches.py:19, in patch_matplotlib.<locals>.show(block)
     17 def show(*, block=None):
     18     buf = BytesIO()
---> 19     matplotlib.pyplot.savefig(buf, format="png")
     20     buf.seek(0)
     21     display(Image(buf.read()))

...

File /lib/python3.10/site-packages/matplotlib/backends/wasm_backend.py:21
     19 from matplotlib.backend_bases import FigureManagerBase, _Backend
     20 from matplotlib.backends import backend_agg
---> 21 from matplotlib.backends.browser_backend import FigureCanvasWasm, NavigationToolbar2Wasm
     23 from js import ImageData, document
     25 interactive(True)

File /lib/python3.10/site-packages/matplotlib/backends/browser_backend.py:5
      1 import math
      3 from matplotlib.backend_bases import FigureCanvasBase, NavigationToolbar2, TimerBase
----> 5 from js import document
      6 from pyodide.ffi.wrappers import (
      7     add_event_listener,
      8     clear_interval,
   (...)
     11     set_timeout,
     12 )
     14 try:

ImportError: cannot import name 'document' from 'js' (unknown location)

Note: This does not happen in Jupyter Lab (at least not using the recent jupyter/scipy-notebook docker image).

If you un-comment line 2 or comment line 3 of the original example, your session will be fine.

[bollwyvl]

Yeah, it looks like the pyodide-distributed wasm_backend assumes it's running in a main thread pyodide, which is not the case on pyolite, where everything is running in a webworker: if it wasn't it might wedge the whole application in this case, and you might not see anything (or it would only be in the kernel).

It looks like this choice was added semi-recently: https://github.com/pyodide/pyodide/pull/3061

We'll have to revisit the patches to ensure that they force a backend that we expect.

[posita]

Thanks @bollwyvl! Please let me know if you discover that this should be filed against pyodide instead, and I'll be happy to move it over.

[bollwyvl]

I believe this belongs here, and is something we can fix in our patch: we had been relying on setting the MPLBACKEND environment variable, but it appears that gets overloaded by something explicitly happening in pyodide now.

After finding the issue and updating the patch, as part of declaring it "fixed", we need to get a minimum reproducer for custom rcParams, which are pretty common, into the matplotlib example notebook, as we run through each of those before doing a pyodide release.