Is there a way of embedding a matplotlib figure in a Remi window? Does not have to be interactive

[MikeTheWatchGuy]

I was able to put a Matplotlib drawing into a PySimpleGUI tkinter window, but with 3.1.3 that broke. I'm working on adding it back.... which got me thinking that I should add it to PySimpleGUIWeb too while I'm at it.

Do you know how to embed the image from a Matplotlib drawing?

The way I did it before it was a static image and it used tkinter specific calls.

[dddomodossola]

Hello @MikeTheWatchGuy here is an example for you https://github.com/dddomodossola/remi/blob/master/examples/matplotlib_app.py . I'm sure You will be able to embed it in PySimpleGuiWeb in minutes. I'm away for work reasons for a week and I've not my computer with me, so I'm unable to program stuffs, until next monday.

[PySimpleGUI]

I've been working this weekend and today on the example application.

Would it be possible for you to create something that doesn't create this special Matplotlib class?

With my tkinter and other ports, I create a window with an "Image Element" or a Canvas, and then the matplotlib code drawing onto the image element.

I would like the exact same kind of thing for the Remi port. I create an image and draw the matplotlib figure onto it.

Here's an example of doing it with the tkinter port:

#!/usr/bin/env python
import numpy as np
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
import PySimpleGUI as sg
import matplotlib
matplotlib.use('TkAgg')

"""
Demonstrates one way of embedding Matplotlib figures into a PySimpleGUI window.

Basic steps are:
 * Create a Canvas Element
 * Layout form
 * Display form (NON BLOCKING)
 * Draw plots onto convas
 * Display form (BLOCKING)

 Based on information from: https://matplotlib.org/3.1.0/gallery/user_interfaces/embedding_in_tk_sgskip.html
 (Thank you Em-Bo & dirck)
"""

fig = matplotlib.figure.Figure(figsize=(5, 4), dpi=100)
t = np.arange(0, 3, .01)
fig.add_subplot(111).plot(t, 2 * np.sin(2 * np.pi * t))

# ------------------------------- END OF YOUR MATPLOTLIB CODE -------------------------------

# ------------------------------- Beginning of Matplotlib helper code -----------------------

def draw_figure(canvas, figure):
    figure_canvas_agg = FigureCanvasTkAgg(figure, canvas)
    figure_canvas_agg.draw()
    figure_canvas_agg.get_tk_widget().pack(side='top', fill='both', expand=1)
    return figure_canvas_agg

# ------------------------------- Beginning of GUI CODE -------------------------------

# define the window layout
layout = [[sg.Text('Plot test')],
          [sg.Image(key='-CANVAS-')],
          [sg.Button('Ok')]]

# create the form and show it without the plot
window = sg.Window('Demo Application - Embedding Matplotlib In PySimpleGUI', layout, finalize=True, element_justification='center', font='Helvetica 18')

# add the plot to the window
image_element = window['-CANVAS-']
fig_canvas_agg = draw_figure(image_element.Widget, fig)

event, values = window.read()

window.close()

[PySimpleGUI]

What I would like is another draw_figure function for my Remi port.

I tried doing this, but it's not quite enough:

import PySimpleGUIWeb as sg
import numpy as np
from matplotlib.backends.backend_tkagg import FigureCanvasAgg

import io
import time
import threading

def draw_figure(fig, buflock, widget):
    canv = FigureCanvasAgg(fig)
    buf = io.BytesIO()
    canv.print_figure(buf, format='png')
    with buflock:
        if buf is not None:
            buf.close()
        buf = buf

    i = int(time.time() * 1e6)
    widget.attributes['src'] = "/%s/get_image_data?update_index=%d" % (id(widget), i)

    widget.redraw()

def main():
    import matplotlib.figure

    # ------------------------------- START OF YOUR MATPLOTLIB CODE -------------------------------
    buflock = threading.Lock()

    fig = matplotlib.figure.Figure(figsize=(5, 4), dpi=100)
    t = np.arange(0, 3, .01)
    fig.add_subplot(111).plot(t, 2 * np.sin(2 * np.pi * t))

    layout = [
                [sg.T('Matplotlib Exampe', font='Any 20')],
                [sg.Image(key='-IMAGE-')],
                [sg.B('Go'), sg.B('Exit')],
            ]

    window = sg.Window('Title', layout)

    image_element = window['-IMAGE-']       # type: sg.Image
    while True:
        event, values = window.read()
        if event == 'Exit' or event == sg.WIN_CLOSED:
            break
        draw_figure(fig, buflock, image_element.Widget)

        # image_element.update(filename=r'C:\Python\PycharmProjects\GooeyGUI\logo100.jpg')

    window.close()

if __name__ == "__main__":
    main()

It creates a window like this:

I am capable of drawing images onto that Image element with no problems by commenting out the call to draw_figure and instead calling the Image.update method which draws a file.

[PySimpleGUI]

One clear difference between my code and the sample you provided is that I'm missing this function:

    def get_image_data(self, update_index):
        with self._buflock:
            if self._buf is None:
                return None
            self._buf.seek(0)
            data = self._buf.read()

        return [data, {'Content-type': 'image/png'}]

I see that it's indirectly called in the redraw method:

        self.attributes['src'] = "/%s/get_image_data?update_index=%d" % (id(self), i)

I don't know if you recall, but my Image element is represented by a SuperImage class you made for me.

Here is my latest PySimpleGUIWeb.py file that has this class in it.

It feels like I'm getting kinda close, but I don't know/understand how to get this missing function integrated correctly.

PySimpleGUIWeb.py.txt

[PySimpleGUI]

I got something to work!

import PySimpleGUIWeb as sg
import numpy as np
from matplotlib.backends.backend_tkagg import FigureCanvasAgg
import matplotlib.figure

import io
import threading

buflock = None
buf = None

def draw_figure(fig, element):
    global buf, buflock

    canv = FigureCanvasAgg(fig)
    buf = io.BytesIO()
    canv.print_figure(buf, format='png')

    with buflock:
        if buf is None:
            return None
        buf.seek(0)
        data = buf.read()
        element.update(data=data)

def main():

    global buflock

    # ------------------------------- START OF YOUR MATPLOTLIB CODE -------------------------------
    buflock = threading.Lock()

    fig = matplotlib.figure.Figure(figsize=(5, 4), dpi=100)
    t = np.arange(0, 3, .01)
    fig.add_subplot(111).plot(t, 2 * np.sin(2 * np.pi * t))

    layout = [
                [sg.T('Matplotlib Exampe', font='Any 20')],
                [sg.Image(key='-IMAGE-')],
                [sg.B('Go'), sg.B('Exit')],
            ]

    window = sg.Window('Title', layout, finalize=True)

    image_element = window['-IMAGE-']       # type: sg.Image

    while True:
        event, values = window.read()
        if event == 'Exit' or event == sg.WIN_CLOSED:
            break
        draw_figure(fig, image_element)

    window.close()

if __name__ == "__main__":
    main()

[PySimpleGUI]

Here's about the most simple I can make it:

import PySimpleGUIWeb as sg
import numpy as np
from matplotlib.backends.backend_tkagg import FigureCanvasAgg
import matplotlib.figure
import io

def create_figure():
    # ------------------------------- START OF YOUR MATPLOTLIB CODE -------------------------------
    fig = matplotlib.figure.Figure(figsize=(5, 4), dpi=100)
    t = np.arange(0, 3, .01)
    fig.add_subplot(111).plot(t, 2 * np.sin(2 * np.pi * t))

    return fig

def draw_figure(fig, element):
    canv = FigureCanvasAgg(fig)
    buf = io.BytesIO()
    canv.print_figure(buf, format='png')

    if buf is None:
        return None
    buf.seek(0)
    data = buf.read()
    element.update(data=data)

def main():

    layout = [
                [sg.T('Matplotlib Example', font='Any 20')],
                [sg.Image(key='-IMAGE-')],
                [sg.B('Draw'), sg.B('Exit')],
            ]

    window = sg.Window('Title', layout)

    while True:
        event, values = window.read()
        if event == 'Exit' or event == sg.WIN_CLOSED:
            break
        if event == 'Draw':
            draw_figure(create_figure(), window['-IMAGE-'])
    window.close()

if __name__ == "__main__":
    main()

[PySimpleGUI]

Well, the victory was short lived. It took less than a week before the solution wasn't ultimately 'good enough'.

Now the request is for the Matplotlib plots to be interactive in addition to being drawn. This has been a real challenge in the PySimpleGUI version, but PySimpleGUIWeb is a whole other thing entirely.,

Your solution for Matplotliob, creating an image, was brilliant and enabled me to create a generalized Matplotlib demo that works on all of the ports.

I was able to even create grids of Matlotlib images this way:

But, as I said, others now want to interact with them.

Here is where the issue is being discussed: https://github.com/PySimpleGUI/PySimpleGUI/issues/3057

This demo program was provided as an example of how it can be done with browsers.

import matplotlib as mpl
mpl.use('webagg')
from matplotlib import pyplot as plt

fig,ax = plt.subplots(1)

ptid = ['1','2','3','4','5','6','7','8','9','10']
x = [2,4,5,7,6,8,9,11,12,12]
y = [1,2,3,4,5,6,7,8,9,10]

ax.scatter(x, y, s=30, color='magenta', alpha=0.7, marker='x', picker=3)
ax.set_title('WebAgg Interactive Matplotlib')

for i, ptlbl in enumerate(ptid):
    ax.annotate(ptlbl, (x[i], y[i]),xytext=(5,0),textcoords='offset points',size=8,color='darkslategrey')

def onpick(event):
    index = event.ind[0]
    print('Point Picked = ', ptid[index])

fig.canvas.mpl_connect('pick_event', onpick)

plt.show()

But it crashes for me now so I'm not sure why it's not working now.

[dddomodossola]

@MikeTheWatchGuy I think we can do this way:

1. Show the graphs as images, exactly as we already do;
2. Get the click event on the image with coordinates;
3. Get the figure manager as in this api https://matplotlib.org/3.2.1/api/_as_gen/matplotlib.pyplot.get_current_fig_manager.html#matplotlib.pyplot.get_current_fig_manager
4. Create a MouseEvent ( described here https://matplotlib.org/3.2.1/api/backend_bases_api.html#matplotlib.backend_bases.MouseEvent )
5. Call the matplotlib method pick to simulate a click on the real canvas ( https://matplotlib.org/3.2.1/api/backend_bases_api.html#matplotlib.backend_bases.FigureCanvasBase.pick )

This should trigger the mouse event. I will do my best to test this this evening ;-)