Different IPython interface

[jankatins]

I've found that the svg rendering becomes a real performance hog when the plotted graph gets big.

I've build the following functions, which are enabled similar to the version which manages the image plotting of matplotlib figures:

def render_igraph_plot(plot, fmt):
    """returns a image representation of this plot as a string.

    Parameters
    ==========
    plot : igraph.Plot
        The plot which should be converted
    formats : str or set
        The image format: 'png', 'svg', 'pdf'.
    """
    import igraph
    from igraph.drawing.utils import find_cairo, BoundingBox
    from io import BytesIO

    cairo = find_cairo()
    io = BytesIO()

    width = plot.bbox.width
    height = plot.bbox.height
    orig_bbox = None
    orig_obj = None 

    # This is an ugly hack :-( Unfortunately I see no other way to change 
    # the size of a already existing igraph plot. 
    # We only change the size if this is a "normal" plot, meaning that the 
    # plot has exactly one object to plot and the bbox of this object is a 
    # "standard" one, i.e., size is 2*20 lower than the one from the plot   
    if len(plot._objects) == 1:
        if (plot.bbox.width == plot._objects[0][1].width + 40 and 
            plot.bbox.height == plot._objects[0][1].height + 40):
            orig_bbox = plot.bbox
            orig_obj = plot._objects[0]

            figsize = getattr(igraph, "figsize", (6,4)) # same as matplotlib
            dpi = getattr(igraph, "dpi", 80) # same as matplotlib

            if fmt == "png":
                width = figsize[0] * dpi
                height = figsize[1] * dpi
            else:
                width = figsize[0] * 72
                height = figsize[1] * 72

            new_bbox = BoundingBox(width, height)
            new_obj = (orig_obj[0], new_bbox.contract(20), orig_obj[2], orig_obj[3], orig_obj[4], orig_obj[5])
            plot.bbox = new_bbox
            plot._objects[0] = new_obj

    if fmt == "png":
        surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, int(width), int(height))
    elif fmt == "svg":
        surface = cairo.SVGSurface(io, width, height)
    elif fmt == "pdf":
        surface = cairo.PDFSurface(io, width, height)
    else:
        raise ValueError("Unknown format: %s" % fmt)

    context = cairo.Context(surface)
    plot.redraw(context)

    if fmt != "png":
         # No idea why this is needed but python crashes without
        context.show_page()
        surface.finish()
    else:
        # png is treated differently, the rest has the IO in the surface creation
        surface.write_to_png(io)

    ret = io.getvalue()

    if fmt == "svg":
        # the same does ipython with matplotlib figures
        ret = ret.encode("utf-8")

    if orig_bbox or orig_obj:
        plot.bbox = orig_bbox
        plot._objects[0] = orig_obj

    return ret

def set_igraph_figsize_in_ipython(figsize=(8,6), dpi=80):
    import igraph
    if not isinstance(figsize, tuple):
        raise ValueError("figsize must be a tuple (width, heigh) in inch")

    try: 
        dpi = int(dpi)
    except:
        raise ValueError("dpi must be an int")

    igraph.figsize = figsize
    igraph.dpi = dpi

def select_igraph_plot_formats(formats):
    """Select figure formats for the inline backend.

    Parameters
    ==========
    formats : str or set
        One or a set of figure formats to enable: 'png', 'svg', 'pdf'.
    """
    from IPython.core.interactiveshell import InteractiveShell
    from IPython.utils import py3compat
    from igraph import Plot

    shell = InteractiveShell.instance()

    svg_formatter = shell.display_formatter.formatters['image/svg+xml']
    png_formatter = shell.display_formatter.formatters['image/png']
    pdf_formatter = shell.display_formatter.formatters['application/pdf']

    if isinstance(formats, py3compat.string_types):
        formats = {formats}
    # cast in case of list / tuple
    formats = set(formats)

    [ f.pop(Plot, None) for f in shell.display_formatter.formatters.values() ]

    supported = {'png', 'svg', 'pdf'}
    bad = formats.difference(supported)
    if bad:
        bs = "%s" % ','.join([repr(f) for f in bad])
        gs = "%s" % ','.join([repr(f) for f in supported])
        raise ValueError("supported formats are: %s not %s" % (gs, bs))

    if 'png' in formats:
        png_formatter.for_type(Plot, lambda plot: render_igraph_plot(plot, "png") )
    if 'svg' in formats:
        svg_formatter.for_type(Plot, lambda plot: render_igraph_plot(plot, "svg") )
    if 'pdf' in formats:
        pdf_formatter.for_type(Plot, lambda plot: render_igraph_plot(plot, "pdf") )

Usage:

import igraph
select_igraph_plot_formats(["svg"])
set_igraph_figsize_in_ipython(figsize=(4,3), dpi=80)
g = igraph.Graph.Barabasi(5,6)
plot = igraph.plot(g)
plot

Would you take such a patch?

[ntamas]

Sorry for the late reply, I'm kinda swamped with work in the last few weeks.

Is this patch for IPython? Would this mechanism simply replace the existing _repr_png and _repr_svg methods?

For what it's worth, others have also reported problems with the SVG rendering, although not from the performance side - it seems like the SVG plots embedded in an IPython notebook do not render the vertex labels properly, so I'm very interested in finding a solution that allows the user to at least configure what format the plots are sent to the IPython backend, and I'm happy to make PNG the default if it solves the performance problems.

[jankatins]

No problem.

This could replace the current _repr_svg_() method on Plot. The idea would be to check on importing whether igraph is in an IPython kernel and call select_igraph_plot_formats(["png"]). IPython then calls render_igraph_plot(plot, "png") for the plot object and returns the png-string.

If you want to have a different format, simple call select_igraph_plot_formats(["svg", "pdf"]) and you get both svg and pdf (in this case the notebook just uses the svg output and does not show the pdf, but it's still there in the ipynb file; only PDF and you get a download link for the PDF).

If 'enable on import' is too much, this could also become a user action, much like %matplotlib inline for matplotlib plots.

As a sidenote: if you only want to switch to png, the easiest would be to add this method to Plot:

    def _repr_png_(self):
        """returns a PNG representation of this plot as a string.

        This method is used by IPython to display this plot inline.
        """
        surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, int(self.bbox.width), int(self.bbox.height))
        context = cairo.Context(surface)
        # Plot the graph on this context
        self.redraw(context)

        io = BytesIO()
        surface.write_to_png(io)
        # Return the raw SVG representation
        return io.getvalue()

[EDIT: wrong, it will call all magic methods available]IPython will only call _repr_png_ and not anymore _repr_svg_ AFAIK.[/]

[ntamas]

I have tried an implementation of _repr_png_ before (in response to a complaint from a user where _repr_svg_ did not work for him in a particular setup), but the problem was that adding _repr_png_ alone would not prevent IPython from using SVG in the notebook if _repr_svg_ is already available, and I could not find any easy way to make IPython prefer PNG over SVG, so I dropped the idea because I did not want to remove SVG support entirely. It looks like your solution would make it possible to add support for all three formats (SVG, PNG and PDF) while also making it possible to use PNG only as the default (to solve others' problems with the SVG rendering), so I'm definitely positive about the idea.

I'm on holiday until the end of this week so let me revisit this thread early next week. I definitely like the idea, but I'd rather try to find a solution where we use the standard configuration mechanism of python-igraph (see the igraph.config module) to specify the default format instead of a custom function like select_igraph_plot_formats. This is probably not a big change compared to your solution, and it would also allow us to specify the default figure size in the configuration as well, rendering the set_igraph_figsize_in_ipython function unnecessary.

Anyway, thanks for the idea and the code samples - I'll definitely get back to this and implement it somehow in the next release. If you feel like doing so, you can fork the master branch of the repo and merge your code into the forked branch, and then I'll revisit your branch once I'm back from holidays.

[jankatins]

I won't have time to work on this until friday next week, so no problem...

Ok, just looked it up: IPython will call all magic methods which are available (so _repr_png_ and _repr_svg_ and whatever else there is) and return the result (so the notebook has all formats available, but only displays one).

One way to work around this implementing something that makes getattr(plot_object, "_repr_svg_", None) return None if svg is not the prefered format.

Oh, and this should also work: raise a NotImplementedError in _repr_svg_ if svg should not be computed. The @catch_format_error decorator around IPython.core.formatters.BaseFormatter.__call__() should filter that away...

But I feel that the above way with an explicit formatter for Plot is a much nicer and less hackish way than these two solutions...

The main obstacle in setting image sizes is that currently the final size is added to the plot during plot(...) and not during rendering, which resulted in this horrible hack. A better solution should probably add margins to the plot object list and compute the bbox during rendering...

[jankatins]

So, would it be acceptable to remove _rep_png_ and use the formatter way? And if the latter: enable it on import or should that be an explicit step?

[ntamas]

Yes, removing _repr_png_ is acceptable. It would be great if we could somehow detect during import whether we are running inside IPython and then set up the plotting automatically.

[jankatins]

Another question: I would like to remove the hack in the patch by pushing the size calculations from the plot() function into the methods of the Plot object. this might result in code duplication but then size will be determined in the actual drawing and not before. IMO this will not change the behaviour in non IPython environments and IMO also not in IPython environments. Is this also acceptable?

[ntamas]

Yes, it is also acceptable.

[iosonofabio]

Now that matplotlib is an accepted plotting engine for python-igraph, you can use its PNG (or other) static raster backends to achieve this. I'd be for closing this one and telling people to use mpl. But @ntamas this is just my personal preference.

[ntamas]

Let's go back to the beginning of this thread. If I understand correctly, the original motivation for this patch was this:

I've found that the svg rendering becomes a real performance hog when the plotted graph gets big.

The question is: is this still a problem, and if it is, is this solved with the Matplotlib backend? If Matplotlib solves this, then we can safely close this issue. Otherwise we should attempt to find a solution somehow, first by figuring out where the performance bottleneck is. From a cursory look at the original proposal, it still draws the graph through Cairo by the usual means but avoids jumping through a few hoops, so the original performance bottleneck is somewhere in the machinery around the Cairo plotting and not in Cairo itself.

[iosonofabio]

Might be worth mentioning that we are considering switching to matplotlib as default. Mpl deals with backends (e.g. PNG/SVG) quite flexibly and is generally quite well integrated with notebooks and ipython as well, so if we transition we might end up in the same boat as many other libraries in terms of performance: might not support billions of nodes, but it will let the user choose the backend at will.