matplotlib / ipympl

Matplotlib Jupyter Integration
https://matplotlib.org/ipympl/
BSD 3-Clause "New" or "Revised" License
1.57k stars 226 forks source link

Re-executing a cell hides the figure when it is given a `num=...` argument #405

Open tovrstra opened 2 years ago

tovrstra commented 2 years ago

Describe the issue

Explicit figure numbers seem to cause a small glitch in ipympl. When the following cell is executed once, it shows the plot. When re-executing the cell, it disappears again.

x = np.linspace(-np.pi/2, np.pi/2)
plt.figure(num=1)
plt.plot(x, np.sin(x))

I've attached a working example: repeat_plot_issue.zip

The inline and notebook backends do not have this issue, which leads me to believe this is a bug. There are two workarounds:

  1. Close the figure in the beginning of the cell with e.g. plt.close(1)
  2. Show the figure at the end of the cell with plt.show()

(I'm using Jupyter notebooks with a group of about 40 students, and they got confused by this difference in behavior between the backends. I understand this is not a major issue because there are simple workarounds. Still, for newcomers, it is an extra difficulty: "Help, my plot has disappeared!")

Versions

 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:20:46) 
[GCC 9.4.0]
ipympl version: 0.8.2
Selected Jupyter core packages...
IPython          : 7.30.1
ipykernel        : 6.6.0
ipywidgets       : 7.6.5
jupyter_client   : 6.1.12
jupyter_core     : 4.9.1
jupyter_server   : 1.13.1
jupyterlab       : 3.2.5
nbclient         : 0.5.9
nbconvert        : 6.3.0
nbformat         : 5.1.3
notebook         : 6.4.6
qtconsole        : not installed
traitlets        : 5.1.1
Known nbextensions:
  config dir: /home/toon/.jupyter/nbconfig
    notebook section
      splitcell/splitcell disabled
      nbextensions_configurator/config_menu/main  enabled 
      - Validating: problems found:
        - require?  X nbextensions_configurator/config_menu/main
      contrib_nbextensions_help_item/main  enabled 
      - Validating: OK
      spellchecker/main  enabled 
      - Validating: OK
      toc2/main  enabled 
      - Validating: OK
      k3d/extension  enabled 
      - Validating: OK
    tree section
      nbextensions_configurator/tree_tab/main  enabled 
      - Validating: problems found:
        - require?  X nbextensions_configurator/tree_tab/main
  config dir: /home/toon/.local/etc/jupyter/nbconfig
    notebook section
      nbdime/index  enabled 
      - Validating: problems found:
        - require?  X nbdime/index
      rise/main  enabled 
      - Validating: OK
  config dir: /home/toon/miniconda3/envs/py4sci/etc/jupyter/nbconfig
    notebook section
      jupyter-matplotlib/extension  enabled 
      - Validating: OK
      rise/main  enabled 
      - Validating: OK
      voila/extension  enabled 
      - Validating: OK
      jupyter-js-widgets/extension  enabled 
      - Validating: OK
      nbextensions_configurator/config_menu/main  enabled 
      - Validating: problems found:
        - require?  X nbextensions_configurator/config_menu/main
      contrib_nbextensions_help_item/main  enabled 
      - Validating: OK
    tree section
      nbextensions_configurator/tree_tab/main  enabled 
      - Validating: problems found:
        - require?  X nbextensions_configurator/tree_tab/main
  config dir: /etc/jupyter/nbconfig
    notebook section
      jupyter-js-widgets/extension  enabled 
      - Validating: OK
JupyterLab v3.2.5
/home/toon/miniconda3/envs/py4sci/share/jupyter/labextensions
        k3d v2.11.0 enabled OK (python, k3d)
        jupyter-matplotlib v0.10.2 enabled OK
        @jupyter-widgets/jupyterlab-manager v3.0.1 enabled OK (python, jupyterlab_widgets)
        @ijmbarr/jupyterlab_spellchecker v0.7.2 enabled OK (python, jupyterlab-spellchecker)
        @ryantam626/jupyterlab_code_formatter v1.4.10 enabled OK (python, jupyterlab-code-formatter)
        @voila-dashboards/jupyterlab-preview v2.1.0 enabled OK (python, voila)
martinRenou commented 2 years ago

Thanks for reporting this issue.

For some reason, the issue doesn't appear if the activation of the backend is done in the same cell:

%matplotlib widget

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(-np.pi/2, np.pi/2)
plt.figure(num=4)
plt.plot(x, np.sin(x));
ianhi commented 2 years ago

This got kinda long the tldr is: I think this is actually a bug in nbagg and the best solution is probably to teach your students to use the explicit (nee OO) interface like so:

fig1, ax1 = plt.subplots()
ax1.plot(...)

The inline and notebook backends do not have this issue, which leads me to believe this is a bug. There are two workarounds:

Inline probably isn't the best comparison because it displays and then closes all figures at the end of every cell. I'm a little bit surprised by the behavior of nbagg though. As I look at this more though I think this may actually be a bug in nbagg?

My understanding of the behavior of plt.figure(num=1) is that it will generate and display(when in a notebook) a new figure if no figure with ID=1 exists, otherwise it will set the current figure (plt.gcf()) and not do any further display. It seems like the nbagg backend is regenerating the frontend every time plt.figure is called, while ipympl is, I think correctly, recognizing that the figure already exists and now doing any more display or creation logic. Then the outputs of the cell are overwritten as expected by the new execution.

For example if you run the following two cells:

cell 1

%matplotlib nbagg
import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(-np.pi/2, np.pi/2)
i=0

cell 2 (run this one repeatedly)

plt.figure(num=1)
plt.plot(x, np.sin(x*i))
i += 1

then you only ever have a plot with a single line in it.

nbagg-bug

but I think the correct behavior should be to have multiple lines show up. To get that you can do the equivalent from ipython script

import matplotlib.pyplot as plt
import numpy as np

plt.ion()
x = np.linspace(-np.pi/2, np.pi/2)

# each iteration of loop represents a cell re-execution
i = 0
def run_cell():
    global i
    plt.figure(num=1)
    plt.plot(x, np.sin(x*i))
    i += 1

correct-ipython

For some reason, the issue doesn't appear if the activation of the backend is done in the same cell:

I think this is because setting the backend closes all figures so you are no longer refering to the same figure.

ianhi commented 2 years ago

This got kinda long the tldr is: I think this is actually a bug in nbagg and the best solution is probably to teach your students to use the explicit (nee OO) interface like so:

That's a bit short of an explanation - happy to talk more about this if I can be helpful!

tovrstra commented 2 years ago

@ianhi Thank you for all the insightful comments! I'm going to experiment with this, before asking more questions.

tacaswell commented 2 years ago

As @jklymak and @QuLogic pointed out in https://github.com/matplotlib/matplotlib/issues/21957 the issue is with the life cyle management of the implicit figures.

In the case of nbagg we have logic that when the js side is removed from view we "close" the figure and drop it from Matplotlib's registry of open figures. From the behavior I think the order of things is output is cleared -> call back from js-to-kernel is run to close the figure -> cell is run -> matplotlib finds there is no figure 1 makes a new one -> new figures are shown at the end of cell execution.

In the IPython + GUI case because the window is never closed, each time through the loop we find that after the first pass there exists a figure 1, plt.figure(num=1) sets it to be the "current figure" and the plt functions operate on its current axes. In the last gif from @ianhi if he closed the window by clicking the 'x' between calling run_cell() (or added plt.close(1) you would get the same behavior as nbagg.

As @ianhi notes with inline it always closes the figure after rendering the cell so it always makes a new one.

In both the nbagg and ipympl cases if you create the figure in one cell, and then repeatedly do plt.figure(num=1) in a subsequent cell, you will get the GUI like behavior of adding lines to the same plot.

Of these semantics, it is not clear which one is "right" and how this should interact with having multiple views of the same output.

Also see the discussion in https://github.com/matplotlib/ipympl/issues/171

tovrstra commented 2 years ago

After trying a few things, I found nbagg's behavior more intuitive. I'll summarize the situation for the common use case of someone refining their plotting code and re-executing that incrementally improved code cell several times.

Comparing all these, a modification of ipympl's behavior to match that of nbagg, seems ideal, but I understand that that is a lot of work.

For the time being, calling plt.close before making any plot seems to be the safest workaround. This always works with any backend or matplotlib API: it avoids an increasing number of unused figures and makes sure the figure is always shown.

tacaswell commented 2 years ago

completely off-topic: the explicit (aka Object Oriented) API is "new" as of ~15 years ago ;)


Try:

fig, ax = plt.subplots(num=1, clear=True)
ax.plot([0, 1], [2, 3])
plt..show()

instead.

tovrstra commented 2 years ago

Thanks for your help! That works indeed, but I'm still puzzled by how ipympl decides what to show when plt.show() is called. The documentation of plt.show mentions that this function shows all open figures. Ipympl seems to work differently (for good reasons). I tried different ways of using it, but did see any pattern yet. Is there some simple rule to understand what is shown by plt.show?

In comparison, the behavior of calling plt.close upfront is more predictable. You always get to see the figure, irrespective of what happened elsewhere in the notebook. It is also a bit clumsy, though, because previously shown widgets for the same figure will stop working.

P.S. Yes, "very old" versus "old" would have been more fitting. :]