Julia plots in emacs buffer

dahtah commented 4 years ago

This is a small experiment to address issue #15. It uses Julia's display mechanism to make plots appear in an emacs buffer. There are two variants: by default, a single plot is shown in the buffer. The alternative variant inserts plots one after the the other, so that one may go back in history a la RStudio (which I think is a nice feature). Right now this is limited to Plots.jl, extending to other packages should not be difficult as long as they can export to svg. How to use:

]add Plots
start a Julia session (C-c C-z).
M-x julia-snail-plot-in-emacs (wait a bit)
then plot(sin,0,2\pi) and the plot should appear in its own buffer To test the other variant: (setq julia-snail-single-plot nil). Plots will start appearing one after the other in the julia plots buffer. Hit M-x julia-snail-plot-in-emacs to revert back to the default display mode.

Some notes on the implementation:

Right now I use emacsclient to send the SVG data as a string, which is not ideal. We should use the julia-snail connection, but it requires large changes to julia-snail I wasn't comfortable making
SVG support on my build of emacs is not very good. It seems to go through imagemagick and is slow. Zooming in and out takes ages. Maybe other emacs builds do better? There's of course the option of sending the plot to emacs as PNG instead.
Once we have this working it should be very easy to re-use the same mechanism to export other objects to emacs, for instance using display(mat) to examine a matrix in org-table mode
Interactivity is a pretty serious limitation at the moment. Can't see how to enable it short of having a browser that works inside emacs (I don't think eww is going to cut it).

The larger questions relate to UI design. I'm not a big fan of plots showing up in the REPL, I think it clutters things unnecessarily. Having things in a separate buffer seems more useful to me. I also don't really know how to handle UI in emacs, short of calling pop-to-buffer to make the plot appear. I'd be satisfied with something resembling the RStudio plot pane: forward/back buttons, zoom and pan, that's it. An option to export the plot maybe, or easy copy/paste.

Looking forward to comments/feedback

gcv commented 4 years ago

This is super-cool. Thanks! I'm pretty swamped so I can't guarantee I'll get to it today or tomorrow, but I promise to review this PR soon.

dahtah commented 4 years ago

All right, thanks, no hurry!

gcv commented 4 years ago

Using emacsclient to make Emacs respond to an event is such a deliciously evil hack. I absolutely love it, but we should not ship it. :) I think it's possible to extend the process filter to do what you need (currently julia-snail--server-response-filter but should be renamed if it starts supporting asynchronous commands from the Julia side!) I will start looking into it in the next few days.

Second, about SVG: what about generating a PNG as a tempfile and then making sure we delete it? I haven't looked into Julia's tempfile APIs, but we need one that allows us to mark it as "delete after closing", along the lines of Python's tempfile.NamedTemporaryFile with delete=True. Then we pass the tempfile's path to Emacs, which opens and displays it. Julia waits for Emacs to open the file, then closes it. Then, when Emacs releases the file handle, it should be automatically deleted by the OS.

For interactivity, quite honestly we don't have much. Emacs doesn't sport too many fancy UI widgets. Maybe you can probably hack something together for following mouse events over the generated image and implement zoom and stuff that way. For the first shippable version, I suggest we stick to pop-to-buffer.

dahtah commented 4 years ago

Thanks for your feedback. I took a look at ElectronDisplay.jl, which inspired some changes. Now all packages that can output png/svg should work (I tested Gadfly, VegaLite). The new default is to send PNG data to emacs as a string (using Base64 encoding). In addition, there's now a minor mode for the plot buffer. The only feature for the moment is that hitting "q" takes you back to the REPL.

Regarding the points you made:

Fully agree that the current emacsclient hack should be replaced before shipping. If you can make the changes necessary to the server code that'd be great. I think it could be useful for other things in the future (for instance, we could hijack the display method for stack traces so that they are shown in an emacs window).
I've thought about use temp files, but I was hoping to avoid needless disk access. It seems wasteful when we can just pipe a bit of data. Maybe that's naive? I know nothing about systems programming. SVG seemed like a good choice to me at first because it's a vector format, enabling zooming and panning on the emacs end of things. Unfortunately image support isn't very good from what I can tell (zooming in on an SVG image in image-mode just rescales a low-res raster rendering, for instance). Another reason not to use SVG is that it's wasteful on large datasets. However, one problem at the moment is that every package has its own default size for PNGs (Plots.jl has actually one per back-end) and they're often way too small. Vegalite.jl's default is the size of half a stamp.
Agreed

dahtah commented 4 years ago

Just for fun, I added a display function for stackframes: now hitting stacktrace() in the REPL will display the stack trace in an Emacs buffer with highlighting. I think it's actually pretty useful for long stack traces.

orialb commented 4 years ago

Just wanted to report that I tried this with PyPlot.jl and it seems that some extra things need to be configured for PyPlot in order for the plots to be opened in an emacs buffer. By default PyPlot opens up an external qt GUI window when called from the REPL (unrelated to emacs), which actually allows zooming and stuff like this. But sometimes it can be nicer to have it in a buffer inside emacs instead. There is a way to disable the GUI by calling pygui(false), but in this case it seems that nothing is plotted. Maybe because PyPlot.plot returns a PyObject, I'll have to look into this more.

Also I noticed that calling julia-snail-plot-in-emacs works only when called from the Snail REPL buffer. When it is called in the source code buffer it gives some error:

(wrong-type-argument stringp nil)
  set-buffer(nil)
  (save-current-buffer (set-buffer process-buf) (goto-char (point-max)) (insert msg))
  (let* ((process-buf (get-buffer (julia-snail--process-buffer-name repl-buf))) (module-ns (julia-snail--construct-module-path module)) (reqid (format "%04x%04x" (random (expt 16 4)) (random (expt 16 4)))) (msg (format "(ns = %s, reqid = \"%s\", code = %s)\n" module-ns reqid (json-encode-string str))) (res nil)) (save-current-buffer (set-buffer process-buf) (goto-char (point-max)) (insert msg)) (process-send-string process-buf msg) (spinner-start (quote progress-bar)) (puthash reqid (record (quote julia-snail--request-tracker) repl-buf (current-buffer) (function (lambda (&optional data) (if async nil (setq res (or data :nothing))) (if callback-success (progn (funcall callback-success data))))) (function (lambda nil (if async nil (setq res :nothing)) (if callback-failure (progn (funcall callback-failure))))) display-error-buffer-on-failure\? nil) julia-snail--requests) (if async reqid (let ((g54 0) (g55 async-poll-interval) (g56 async-poll-maximum)) (while (and (< g54 g56) (null res)) (sleep-for 0 g55) (setq g54 (+ g54 g55)))) res))
  (progn (if repl-buf nil (user-error "No Julia REPL buffer %s found; run julia-snail" julia-snail-repl-buffer)) (let* ((process-buf (get-buffer (julia-snail--process-buffer-name repl-buf))) (module-ns (julia-snail--construct-module-path module)) (reqid (format "%04x%04x" (random (expt 16 4)) (random (expt 16 4)))) (msg (format "(ns = %s, reqid = \"%s\", code = %s)\n" module-ns reqid (json-encode-string str))) (res nil)) (save-current-buffer (set-buffer process-buf) (goto-char (point-max)) (insert msg)) (process-send-string process-buf msg) (spinner-start (quote progress-bar)) (puthash reqid (record (quote julia-snail--request-tracker) repl-buf (current-buffer) (function (lambda (&optional data) (if async nil (setq res ...)) (if callback-success (progn ...)))) (function (lambda nil (if async nil (setq res :nothing)) (if callback-failure (progn ...)))) display-error-buffer-on-failure\? nil) julia-snail--requests) (if async reqid (let ((g54 0) (g55 async-poll-interval) (g56 async-poll-maximum)) (while (and (< g54 g56) (null res)) (sleep-for 0 g55) (setq g54 (+ g54 g55)))) res)))
  (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) (quote (:repl-buf :async :async-poll-interval :async-poll-maximum :display-error-buffer-on-failure\? :callback-success :callback-failure :allow-other-keys))) (setq --cl-keys-- (cdr (cdr --cl-keys--)))) ((car (cdr (memq ... --cl-rest--))) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:repl-buf :async :async-poll-interval :async-poll-maximum :display-error-buffer-on-failure? :callback-success :callback-failure)" (car --cl-keys--)))))) (progn (if repl-buf nil (user-error "No Julia REPL buffer %s found; run julia-snail" julia-snail-repl-buffer)) (let* ((process-buf (get-buffer (julia-snail--process-buffer-name repl-buf))) (module-ns (julia-snail--construct-module-path module)) (reqid (format "%04x%04x" (random (expt 16 4)) (random (expt 16 4)))) (msg (format "(ns = %s, reqid = \"%s\", code = %s)\n" module-ns reqid (json-encode-string str))) (res nil)) (save-current-buffer (set-buffer process-buf) (goto-char (point-max)) (insert msg)) (process-send-string process-buf msg) (spinner-start (quote progress-bar)) (puthash reqid (record (quote julia-snail--request-tracker) repl-buf (current-buffer) (function (lambda (&optional data) (if async nil ...) (if callback-success ...))) (function (lambda nil (if async nil ...) (if callback-failure ...))) display-error-buffer-on-failure\? nil) julia-snail--requests) (if async reqid (let ((g54 0) (g55 async-poll-interval) (g56 async-poll-maximum)) (while (and (< g54 g56) (null res)) (sleep-for 0 g55) (setq g54 (+ g54 g55)))) res))))
  (let* ((repl-buf (car (cdr (or (plist-member --cl-rest-- (quote :repl-buf)) (list nil (get-buffer julia-snail-repl-buffer)))))) (async (car (cdr (or (plist-member --cl-rest-- (quote :async)) (quote (nil t)))))) (async-poll-interval (car (cdr (or (plist-member --cl-rest-- (quote :async-poll-interval)) (quote (nil 20)))))) (async-poll-maximum (car (cdr (or (plist-member --cl-rest-- (quote :async-poll-maximum)) (list nil julia-snail-async-timeout))))) (display-error-buffer-on-failure\? (car (cdr (or (plist-member --cl-rest-- (quote :display-error-buffer-on-failure\?)) (quote (nil t)))))) (callback-success (car (cdr (plist-member --cl-rest-- (quote :callback-success))))) (callback-failure (car (cdr (plist-member --cl-rest-- (quote :callback-failure)))))) (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) (quote ...)) (setq --cl-keys-- (cdr ...))) ((car (cdr ...)) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:repl-buf :async :async-poll-interval :async-poll-maximum :display-error-buffer-on-failure? :callback-success :callback-failure)" (car --cl-keys--)))))) (progn (if repl-buf nil (user-error "No Julia REPL buffer %s found; run julia-snail" julia-snail-repl-buffer)) (let* ((process-buf (get-buffer (julia-snail--process-buffer-name repl-buf))) (module-ns (julia-snail--construct-module-path module)) (reqid (format "%04x%04x" (random ...) (random ...))) (msg (format "(ns = %s, reqid = \"%s\", code = %s)\n" module-ns reqid (json-encode-string str))) (res nil)) (save-current-buffer (set-buffer process-buf) (goto-char (point-max)) (insert msg)) (process-send-string process-buf msg) (spinner-start (quote progress-bar)) (puthash reqid (record (quote julia-snail--request-tracker) repl-buf (current-buffer) (function (lambda ... ... ...)) (function (lambda nil ... ...)) display-error-buffer-on-failure\? nil) julia-snail--requests) (if async reqid (let ((g54 0) (g55 async-poll-interval) (g56 async-poll-maximum)) (while (and ... ...) (sleep-for 0 g55) (setq g54 ...))) res)))))
  julia-snail--send-to-server(:Main "pushdisplay(JuliaSnail.EmacsDisplay());" :repl-buf #<buffer tmp.jl> :async nil :callback-success (lambda (&optional _data) (progn (message "Plotting inside Emacs turned on") (setq julia-snail--plotting t))))
  (progn (server-start) (julia-snail--send-to-server :Main "pushdisplay(JuliaSnail.EmacsDisplay());" :repl-buf buf :async nil :callback-success (function (lambda (&optional _data) (progn (message "Plotting inside Emacs turned on") (setq julia-snail--plotting t))))))
  julia-snail--init-plotting(#<buffer tmp.jl>)
  (if julia-snail--plotting (julia-snail--cancel-plotting (current-buffer)) (julia-snail--init-plotting (current-buffer)))
  julia-snail-plot-in-emacs()
  funcall-interactively(julia-snail-plot-in-emacs)

By the way, maybe a better name for julia-snail-plot-in-emacs could be julia-snail-toggle-plotting-in-emacs or something like this? From the current name it might seem like this command is actually going to plot something.

In any case, thanks for your work on this feature!

dahtah commented 4 years ago

@orialb Thanks! PyPlot.jl should define the following: showable("image/png",Figure) = true This lets you view the current plot inside emacs using gcf(). It's worth reporting the issue to PyPlot.jl authors so that we don't have to include a conditional hook here. I fixed the other two issues. Thanks for your feedback!

gcv commented 4 years ago

@dahtah: Here's the API function you need to replace emacsclient: JuliaSnail.send_to_client. See e2f1e60. I pushed it to master, so either rebase on top of it or merge it in.

For your use, omit the socket parameter. It's there for the not-quite-working-right multiple-clients feature.

Once you make your changes, I'll take another look at your work.

dahtah commented 4 years ago

Thanks! Done.

benide commented 4 years ago

Good work! A couple comments:

I really dislike the behavior where the plot window takes focus (though I realize this is how it works when you just use a REPL). I will just change it with shackle.el, but it might be something worth considering for the default behavior.
For anyone using straight.el, you need to make sure your recipe includes a file directive for the julia files. The EmacsDisplay.jl will not be symlinked if you don't do this. The recipe I'm using with use-package is:
```
:straight (julia-snail :files (:defaults "JuliaSnail.jl" "EmacsDisplay.jl")
         :host github
         :repo "gcv/julia-snail"
         :fork (:host github :repo "dahtah/julia-snail" :branch "snail-plots"))
```

I haven't used it a ton yet, but I'll incorporate it into my workflow tomorrow and let you know how it goes :)

dahtah commented 4 years ago

@benide thanks for the feedback. Regarding the pop-buffer behaviour you dislike, I can make it optional. What would you recommend as an alternative?

gcv commented 4 years ago

Whatever you do must remain user-configurable with display-buffer-alist.

benide commented 4 years ago

The behavior I'd want is that the *julia plot* buffer gets updated without getting selected when I plot. If I plotted directly from the REPL, I'd like to still be in the REPL. If I sent a line to the REPL from a Julia source buffer, I'd like to stay in the source buffer. I can't actually think of anything I'd do from the plot buffer. What's a scenario where you're actually wanting your current buffer to be *julia plot*?

But, I want to stress, this is awesome and thank you gcv for snail and dahtah for the plotting features :D

edit: Figured I should be a little clearer about what my workflow looks like. I set up the windows in my frame like this:

 _______________
|        | plot |
|  main  |______|
|        | repl |
|________|______|

My plots have a lot of info that help inform what I'm doing in the main window, and I sometimes plot fairly frequently as I'm working on a model, but at no point in my workflow do I feel the need to hit "q" in the plot buffer, which is the only thing I can think of that can be done from there.

gcv commented 4 years ago

Using display-buffer instead of pop-to-buffer should do that. It sounds like the right thing.

dahtah commented 4 years ago

Latest version now uses display-buffer as default, let me know if that helps

gcv commented 4 years ago

No time yet to review in detail, but it looks like we need to update the MELPA recipe to allow multiple *.jl files in the package. I opened a PR to get that ball rolling: https://github.com/melpa/melpa/pull/6903

gcv commented 3 years ago

At very very long last, this code is in! Commit 2753be5. I didn't end up merging in this PR, but the implementation owes a great deal to your work and ideas here, @dahtah. Thank you so much!

gcv / julia-snail

Julia plots in emacs buffer #21