Layout Templates and matplotlib figures

[thomasaarholt]

see #117 (issue) and #174 (PR) for some discussion on the topic.

@martinRenou I have been playing with using Layout Templates to stack figures in fun ways. Unfortunately, since matplotlib is currently responsible for the figure size, I end up with situations such as the following, where the figures are all cut off a bit. This is particularly noticable in jupyter notebook since the screen width is reduced.

In the following example, I stack a 4x4 of buttons and then a 4x4 of figures. The buttons stretch nicely, but the figures do not.

%matplotlib widget
import matplotlib.pyplot as plt
from ipywidgets import TwoByTwoLayout
from ipywidgets.widgets import Button, Layout

# Buttons
def button(name):
    return Button(description = name, layout=Layout(height='auto', width='auto'))

a,b,c,d =  button("1"), button("2"), button("3"), button("4"),
layout_2x2 = TwoByTwoLayout(top_left=a, 
                            top_right=b,
                            bottom_left=c,
                            bottom_right=d)
display(layout_2x2)

# Figures
figs = []
plt.ioff()
for i in range(4):
    fig, ax = plt.subplots()
    ax.imshow([[1,2], [3,4]])
    figs.append(fig.canvas)

a,b,c,d = figs

layout_2x2 = TwoByTwoLayout(top_left=a, 
                            top_right=b,
                            bottom_left=c,
                            bottom_right=d)
display(layout_2x2)

In jupyter lab the problem is less so, but still unfortunate that figures basically ignore the flexbox layout the widgets are designed with. I would set the size with matplotlib, but since I don't know the size of the viewport the user is using, it's likely that the size I choose will be too small or too large depending on the user.

I don't have a concrete solution at hand, but I'm wondering if we can come up with some logic that preserves aspect ratio but otherwise fills a widget to its fullest extent?

[martinRenou]

[...] unfortunate that figures basically ignore the flexbox layout the widgets are designed with

That was the whole point of making the Matplotlib figures responsive, which was the source of issues like https://github.com/matplotlib/ipympl/issues/117.

But in the end, we found that making the Matplotlib figure responsive was not possible. By design, a Jupyter widget can be displayed at multiple places in the Notebook, those multiple places could have different available sizes. And you would end up having two display of the Matplotlib figure, requesting two different sizes to the Matplotlib renderer. This is the reason why it cannot work, why we reverted it and put a resize handle instead.

I don't have a concrete solution at hand, but I'm wondering if we can come up with some logic that preserves aspect ratio but otherwise fills a widget to its fullest extent?

That could be a solution. We could add some kind of flag to the wigdet, something like stretched or responsive. And when this flag is set to True, instead of requesting the renderer a resize of the available space, we would stretch/shrink the image to the available space respecting the ratio. We would lose quality when stretched, but I guess we don't have a better solution now.

[thomasaarholt]

I actually felt really bad bringing this topic up again - I rewrote the issue a few times and considered leaving it as a comment on either the other issue or a PR. I realized that I didn't fully understand the extent of the issue until yesterday.

I like the idea of a flag. If we can preserve aspect ratio while stretching, I think that satisfies most users expectations and desires (though I'm probably being naive). I'm happy to thoroughly test any suggestions.

[martinRenou]

Please don't feel bad! I, myself, felt a bit frustrated we had to revert the changes, but it's the way it is. It would be super great to have a responsive Matplotlib IMO. But that would require changes in Matplotlib itself I guess.

I like the idea of a flag. If we can preserve aspect ratio while stretching, I think that satisfies most users expectations and desires (though I'm probably being naive). I'm happy to thoroughly test any suggestions.

Yeah, I think this would be a nice addition indeed. But it should be disabled by default I guess.

[thomasaarholt]

Please don't feel bad!

Good :) Just thought I'd be extra polite since this is a lot of work!

Yeah, I think this would be a nice addition indeed. But it should be disabled by default I guess.

Agreed! People who use the widget system will in any case have been exposed to the documentation (I hope!) so they should at least know where to find the flag.

[martinRenou]

Yes!

[borod108]

@thomasaarholt thank you so much for bringing this up - I have a very similar issue, I just started using ipympl and the resize from the front end was the most significant features I needed, then suddenly it's gone, I was so frustrated. It would be great if I could use a flag to have the resize work again.

[martinRenou]

I think you are talking about a different issue @borod108. Could you open another issue so we can discuss there about your issue?

[borod108]

@martinRenou perhaps - what I am missing is the small triangle in the corner you could click and drag and then the entire image would grow. Which is not how matplotlib usually behaves, but as far as I understand since you change the canvas it adjusts to the new size (and maybe I am totally wrong :))

[martinRenou]

Could you check that fig.canvas.resizable is set to True?

[borod108]

@martinRenou so I am probably really doing something wrong :) 'Canvas' object has no attribute 'resizable'

[martinRenou]

Could you please show me the output of:

import ipympl

print(ipympl._version.__version__)

Other than that, are you sure you ran %matplotlib widget on top of your Notebook?

[borod108]

I have: %matplotlib ipympl the output is 0.3.3

just tried with %matplotlib widget (after kernel reset) and it is the same.

[martinRenou]

the output is 0.3.3

That is why you don't have the resize handle. It was introduced in 0.5.0 :) But I suggest installing the last version 0.5.6.

[thomasaarholt]

That could be a solution. We could add some kind of flag to the wigdet, something like stretched or responsive. And when this flag is set to True, instead of requesting the renderer a resize of the available space, we would stretch/shrink the image to the available space respecting the ratio. We would lose quality when stretched, but I guess we don't have a better solution now.

I would really like this sort of flag. I was trying to add matplotlib widgets to jupyterlab-sidecar just now, and there it would be really awesome to take advantage of the flexbox style resizing.

[nvaytet]

Maybe I misunderstood something but after reading this issue and #117, it seems the main problem comes from the fact that ipympl is making matplotlib render PNGs in the background and then displaying a png diff (as mentioned in #55 ).

Could any of this be easier if ipympl was using SVGs instead of PNGs to render the figures? @martinRenou ?

[martinRenou]

Hey Neil :)

That's a good question. I guess we could benefit from ipympl supporting SVG outputs, but I suppose such a feature should be an opt-in (as SVG can become huge with many points on a scatter plot, while PNG stays the same size).