Consider adding UI for moving output from console to a markdown document

[akhmerov]

This is merely a suggestion that I thought could be neat, while playing with the combination of a markdown document with an associated console.

It is very easy to send code to the console, however it requires much more work to return the output from console to the markdown document. Right now the user can apply the workflow illustrated below:

This, however wouldn't work with text, and images themselves are not updated due to browser caching. I believe it would be neat if it was possible to automatically (or upon user pressing a keyboard shortcut) to bring the output of the evaluated expression back to the markdown document.

[jasongrout]

Some thoughts:

(a) hook the kernel directly up to the markdown document, and render outputs directly into the markdown document (i.e., don't go through a console)

(b) drag and drop outputs from the kernel?

Either way, I think the best thing is to convert the output messages into something that Markdown understands - perhaps by having a different rendermime with renderers that produce something that makes sense in markdown.

[rgbkrk]

It would be nice if all the image outputs could get written out to a folder like this:

outputs/
  plot-17950c3f-f3ca-4d35-a936-85c0675efae9.png

Then the markdown document would put them inline:

# Hi

from matplotlib import pyplot pyplot.plot([1, 0, 20])

![plot](outputs/plot-17950c3f-f3ca-4d35-a936-85c0675efae9.png)

[Analect]

Could this be a candidate for cell embedded-attachments to contain the image, rather than outputting the image to an outputs/ folder which would then need to somehow get packaged with the notebook?

However, as per #1398, this would require that embedded-attachment functionality be ported over to the notebook running in JupyterLab first.

[rgbkrk]

What I'm proposing is only for markdown files, not for notebooks.

[ellisonbg]

Will try to reply later on this...

Sent from my iPhone

On Nov 10, 2017, at 9:58 AM, Kyle Kelley notifications@github.com wrote:

What I'm proposing is only for markdown files, not for notebooks.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[rgbkrk]

Just trying to keep our lives interesting here Brian. 😉

[ellisonbg]

I like interesting!!!

This type of functionality is one of the main motivations of JupyterLab, so overall +1.

I think it is important to separate the file format questions from the frontend model/view layer. I agree that in some cases you really want to save a visualization to a png/jpg and put a link to that in the markdown, but that is too specialized a usage case to be the only way of "rendering markdown" in JupyterLab. The biggest issue is that it doesn't provide an easy path to embedding arbitrary rich output (VegaLite, Widets, Plotly, etc.).

The way I have been thinking about it is this...

The simplest way would be to provide a "Run all and render markdown" that extracts all the code blocks from a markdown document , runs them all, and then renders the result with inline inputs. The different classes we have in JupyterLab should make this fairly straightforward. One approach would be to covert the markdown document to a read only notebook model on the fly and then just render+run all on that model. This shouldn't be too much work and would immediately work with all output types.

A more sophisticated approach would be to do some deeper integration of a markdown text file, a code console and rendered markdown. In this case we could probably render the markdown initially without outputs, and then inject the outputs in the right place as you run full code blocks in the code console. The challenge is what to do when the users 1) runs part of blocks or 2) runs things out of order (that is probably easier).

The last thing to think about would be to provide some sort of UI that allows a user to pick a particular rendered output of a given MIME type (that markdown can handle) and then save out the file and put the link into the markdown document. My only point is that this should be a separate step to the running and rendering in JupyterLab.

[akhmerov]

I think it is important to separate the file format questions from the frontend model/view layer. I agree that in some cases you really want to save a visualization to a png/jpg and put a link to that in the markdown, but that is too specialized a usage case to be the only way of "rendering markdown" in JupyterLab. The biggest issue is that it doesn't provide an easy path to embedding arbitrary rich output (VegaLite, Widets, Plotly, etc.).

I thought the main limitation here is that markdown is a minimal format that can only render a very limited range of outputs. It would still be absolutely cool to compose markdown documents via live code execution, but I don't quite understand how it could be extended to anything beyond text or images (and still stay markdown). Or do you have an idea?

[ellisonbg]

Rather than render the markdown document in one shot, we would partition it into a sequence of code blocks + markdown and then run the code blocks and render their output. Think of the markdown text as the "model" - one view is to render the entire text at once, what I am proposing would be an alternative view (probably with its own intermediate model as well).

On Sun, Nov 12, 2017 at 8:52 AM, Anton Akhmerov notifications@github.com wrote:

I think it is important to separate the file format questions from the frontend model/view layer. I agree that in some cases you really want to save a visualization to a png/jpg and put a link to that in the markdown, but that is too specialized a usage case to be the only way of "rendering markdown" in JupyterLab. The biggest issue is that it doesn't provide an easy path to embedding arbitrary rich output (VegaLite, Widets, Plotly, etc.).

I thought the main limitation here is that markdown is a minimal format that can only render a very limited range of outputs. It would still be absolutely cool to compose markdown documents via live code execution, but I don't quite understand how it could be extended to anything beyond text or images (and still stay markdown). Or do you have an idea?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jupyterlab/jupyterlab/issues/3221#issuecomment-343750566, or mute the thread https://github.com/notifications/unsubscribe-auth/AABr0HkIaMSre7xwufFJoc_ZUORio_Vqks5s1yIxgaJpZM4QYwFc .

-- Brian E. Granger Associate Professor of Physics and Data Science Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger@calpoly.edu and ellisonbg@gmail.com

[rgbkrk]

One idea we could do some exploring with is the <object> tag (note: we probably need to safelist certain types, block others). By default, with types like json, it will embed it as a raw document fragment:

As an aside, this also works with embed but only object is supported well across the browsers:

Basically I'm saying that we could embed outputs into a markdown document in a way that would be invalid, though I'm guessing the document itself would be somewhat awkward:

This is my _example_ document. 

from IPython.display import JSON display(JSON({'a': 3})

<!-- if small enough, handle it raw (should base64 encode it) -->
<object type="application/json" data="data:,{'a': 3}" />
<!-- if large enough --> 
<object type="application/json" data="outputs/17950c3f.json" />

We can't override what renders these objects, though we can certainly pick up when they're in the document and replace them with our richer versions.

[ellisonbg]

I am fine with us exploring new markdown based file formats that uses these abstractions, I would like those explorations to be separate from the frontend view/model as I see them as separate concerns. For example, one might even represent a full jupyter notebook on disk as a markdown file with embedded code blocks in a text editor and render in the Rmarkdown style. Conversely, one might represent a markdown document on disk, with outputs in a separate file, as a notebook in the UI. I think we want to embrace a diversity of UIs and storage formats that interoperate.

On Sun, Nov 12, 2017 at 10:45 AM, Kyle Kelley notifications@github.com wrote:

One idea we could do some exploring with is the