Add support for SVG plots using IPython/Matplotlib

[jabirali]

This pull request resolves issue #20:

• It fixes a bug whereby Matplotlib plots "disappear" in comint-mime buffers if you choose to turn on SVG output in IPython/Matplotlib.
• It also turns on SVG support in IPython by default, since this significantly improves the image quality.

For comparison with #20, the same example looks like this after this PR:

[jabirali]

Additional effect of using SVG for Matplotlib plots: I just discovered that the text color appears to follow the current Emacs theme. So if you use a Matplotlib style sheet with a transparent background (or set rcParams["figure.facecolor"] = (0, 0, 0, 0)), then the plot text automatically changes colors when Emacs does (e.g. useful with auto-dark).

[astoff]

Thanks for the PR. The first change we should certainly merge. The second probably as well, but I would like to understand the consequences of enabling SVG output. Are there any downsides? Also: InlineBackend.figure_formats is a list, what happens when there's more than one element in it?

[jabirali]

Thanks for the quick response!

Thanks for the PR. The first change we should certainly merge.

Great! That one is the most important, since the second change doesn't necessarily have to be in comint-mime itself.

The second probably as well, but I would like to understand the consequences of enabling SVG output.

This is a valid concern, and I agree that it's good to think through the default carefully. I think we have three options:

• Merge the current code, thus changing to SVG plots by default.
• Drop the second line and mention in the comint-mime README that the user can add get_ipython().run_line_magic("config", "InlineBackend.figure_formats = ['svg']") to their IPython config in ~/.config/ipython/profile_default/startup/start.py to get SVG plots. (One potential downside is that presumably users that work over TRAMP would need to sync their IPython configuration to remote hosts to get this feature).
• Ad a user-configurable option like e.g. comint-mime-image-scalable that can be used to request SVG plots from the REPL

After thinking a bit about it (see also the comments below and the revised PR), the last option would perhaps be ideal?

Are there any downsides?

I did a bit of research, and it seems that there are two downsides:

• SVG figures can in some cases be many orders of magnitude larger than the corresponding PNG figures. For this reason, VSCode first deprecated the feature and made it opt-in, and in the end removed it entirely (despite some people still using it).
• The figure text follows the current Emacs theme. While I find this a positive thing, I haven't tested it across a wide range of plots and configurations (e.g. many different Matplotlib themes, inset plots, different operating systems, etc.). Depending on how consistently the SVG text is fontified, the text might become illegible.

I now think that SVG might not be a great default, especially since it would certainly be surprising to a user if Emacs spent gigabytes of memory on showing a few plots. As a user, I would however still enjoy an option like e.g. comint-mime-image-scalable to easily toggle it on, as the SVG looks much significantly better here. I remember trying the SVG support in VSCode before and didn't have any issues, but it probably depends on e.g. the data density, plot type, and hardware.

Also: InlineBackend.figure_formats is a list, what happens when there's more than one element in it?

Based on this, it appears that it is a set and not a list, and that the allowed values in that set are png, retina, jpeg, svg, pdf.

I did a bit of testing on my machine as to what these do:

• ['svg'] is the only combo that produces crisp plots.
• ['png'], ['png', 'svg'], and ['svg', 'png'] all produce the default blurry plots. Likely, these are equivalent to ['png'].
• ['jpg'] is more blurry than ['png'] (as expected).
• Setting it to ['png', 'retina'] produces twice as large plots, but they still look blurry. I think however this is because Emacs on MacOS implements Retina support in a "weird" way (one needs to inform Emacs that a raster image is high-DPI via :data-2x.)
• ['pdf'] makes the image disappear. This is probably due to the same MIME_TYPE issue as we had to fix when using SVG output.

I think the conclusion is the only values that are useful in the context of comint-mime would be the default ['png'] for raster images or ['svg'] for scalable images. Instead of a general comint-mime-image-format type option to choose between these, it might then make sense to have a more specific comint-mime-image-scalable option that toggles whether we should prefer SVG output from REPLs when supported (with currently only Python implemented of course).

[jabirali]

Based on the considerations above, I have now updated the pull request as follows (see c7af585):

• I added a setting comint-mime-image-scalable (boolean) that allows the user to request SVG images instead of PNG images from REPLs. This makes it easy for e.g. me to drop a (setq comint-mime-image-scalable t) into my config to enable SVG plots in IPython/Matplotlib while keeping the default setting at the perhaps more robust PNG value.
• I modified comint-mime-setup-python on the Lisp side and __COMINT_MIME_setup on the Python side to transmit this user setting from Emacs to IPython, and to thus only enable the SVG plot format if the user requests it.
• I also changed the default figure background (i.e. the color of the area around the plot itself) to be transparent when SVG is enabled. That way, it automatically changes color when the user changes their theme.

I think this might perhaps be a better way to implement it than my original PR :).

[astoff]

Thanks for the detailed information. I agree with your conclusion, so I will merge your suggestion. I just have two small requests.

1. How about renaming the variable to comint-mime-prefer-svg?
2. There are some tab characters, could you untabify (replace by spaces)?

I don't need any paperwork to merge this small change, but, because this package is on GNU ELPA, you would need to sign the FSF copyright assignment papers if you want to continue contributing patches.

Also so you know: I will add similar changes in drepl, a new package I'm developing.

[jabirali]

Thanks for the detailed information. I agree with your conclusion, so I will merge your suggestion.

Great, thanks!

I just have two small requests.

1. How about renaming the variable to comint-mime-prefer-svg?

2. There are some tab characters, could you untabify (replace by spaces)?

Done and done: 589ce84. For consistency, I also renamed the corresponding Python variable to prefer_svg.

I don't need any paperwork to merge this small change, but, because this package is on GNU ELPA, you would need to sign the FSF copyright assignment papers if you want to continue contributing patches.

Understood! I don't mind signing the FSF papers, but I haven't done it before so I would have to familiarize myself with the process in that case. I will keep it in mind if I have further suggestions for improvements.

Also so you know: I will add similar changes in drepl, a new package I'm developing.

Sounds good! I'll keep an eye on that project, that sounds interesting as well.

[astoff]

@jabirali FYI, for the reasons explained in #23, I removed the line of your MR that added a workaround for dark theme users; unfortunately it seems in this case the user has to fix the problem on their own.

Thankfully that's not too hard. I added the following explanation to comint-mime-prefer-svg, which you might want to apply:

Using a theme with dark background can render some SVGs unreadable. If you experience this, try setting `comint-mime-image-props' to (:foreground "black" :background "white"). Alternatively, configure the SVG creator to produce images that play well with your theme.

[jabirali]

That sound reasonable. Thanks for the heads up! :)