Design explorations for the notebook document UI

[ellisonbg]

Opening an issue to encourage design explorations of the core notebook document UI, in particular cells. I opened this after folks started exploring some of these ideas in #1692. I am very interested in seeing how we can improve the core cell/notebook UI, both visually and also from a UX perspective.

Here is some initial brainstorming on principles to help guide us in these explorations:

• Fewer borders?
• More solid blocks of color/grey, but with suitable visual contrast
• Visual representation that is "document" like, likely with a strong usage of white (see this paper)
• Efficient and compact usage of vertical space for each cell, so users can work with as many cells as possible.

(feel free to add more principles - even ones that have tension/conflict with these)

@stsievert @ddavidebor @cameronoelsen if you are game I would love help exploring more of these ideas. I love the mockups you did in #1692. I will also start to post mockup as well. Also, the CSS variables in jupyterlab allow very quick exploration of visual ideas as well.

[ddavidebor]

My two cents:

warning: I overthink things.

Borders

I do not think we should necessarily strive for less borders, but we should strive to remove unnecessary ones.

Here is an example:

Contrast:

Perhaps one idea might be a double theme:

• standard fancy looking
• high contrast

prototypicality

Is Jupyter notebook really a notebook? Maybe. But JupyterLab isn't.

The proportions of a notebook are gone, it's just a canvas the user can put where they wants, stretch, put side-by-side...

These two notebooks are not two pieces of paper on top of the other. Not unmanipulated/standard pieces one would use as a notebook.

I have to say it looks a lot like an IDE. (because... it is...). An IDE is a place where you edit files and trigger a compilation of those in the end result.

Now an IDE has a thing that might help usability in JupyterLab. It always has a menu with something like:

• run
• run to debug (shit happened and you need to understand it)
• compile for production

That in jupyterlab could be

• run cell
• run all cells (you want to re-run everything to check it)
• export in formats for the masses (pdf, html...)

Of course this suggestion would make it look less like a document. I don't know if it's what you want, but i think it may be useful to think about how conceptually it could differ.

[stsievert]

a canvas the user can put where they wants, stretch, put side-by-side

Hm... the output cells are really jupyterlab tabs/windows. They can be moved to one side of the screen, in a new tab, etc.

This would be a lot more clear if we restyled the "out" prompt to look like the tabs (and relabeled "mirrored output" to "out [n]"). We could even have the "X" to remove the output for that cell.

[ddavidebor]

0.0 this comes as a surprise to me, I needed something like that just yesterday but had no idea this feature existed.

Definitely there should be some common indicator/look to draggable elements.

[stsievert]

there should be some common indicator/look to draggable elements.

Agreed! I've done a mock up to make output areas look like the tabs:

The vertical whitespace padding is very rough and not accurate, and I'd like to see some of the whitespace on the left removed to focus on content.

I also tried a tab on the left (not the top) but I didn't like it as much.

[ellisonbg]

Keep in mind, each cell can many outputs and at most one will have an Out[] prompt. Also, each individual output can have multiple MIME types (text, HTML, image) and we are thinking about offering a UI for cycle through those MIME types as well.

On Sun, Feb 19, 2017 at 12:05 PM, Scott Sievert notifications@github.com wrote:

there should be some common indicator/look to draggable elements.

Agreed! I've done a mock up to make output areas look like the tabs:

[image: screen shot 2017-02-19 at 1 02 59 pm] https://cloud.githubusercontent.com/assets/1320475/23105368/c7052522-f6a3-11e6-996d-c00ad3ba34a9.png

The vertical whitespace padding is very rough and not accurate, and I'd like to see some of the whitespace on the left removed to focus on content.

I also tried a tab on the left (not the top) but I didn't like it as much.

[image: screen shot 2017-02-19 at 12 32 20 pm] https://cloud.githubusercontent.com/assets/1320475/23105386/efad8b40-f6a3-11e6-9a60-08d2fbbe1f0f.png

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jupyterlab/jupyterlab/issues/1742#issuecomment-280940371, or mute the thread https://github.com/notifications/unsubscribe-auth/AABr0EURdcYquNY3_uBAwqCuM-zO4dqbks5reJJdgaJpZM4MFWEu .

-- Brian E. Granger Associate Professor of Physics and Data Science Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger@calpoly.edu and ellisonbg@gmail.com

[ddavidebor]

@stsievert what about a "grabby" pattern common to tabs and the draggable part of cells?

Something like a vertical strip of grey Braille dots. Altough it's pretty obvious the tabs are draggable...

[ellisonbg]

I like that idea of a grabby pattern. We have been thinking about ways of encoding regions that are draggable, but hadn't gotten very far...nice!

On Sun, Feb 19, 2017 at 2:08 PM, Davide Bortolami notifications@github.com wrote:

@stsievert https://github.com/stsievert what about a "grabby" pattern common to tabs and the draggable part of cells?

[image: braille-975x540] https://cloud.githubusercontent.com/assets/9055742/23106588/e6201ea2-f6ef-11e6-86b5-c2b9f93f91e2.jpg

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jupyterlab/jupyterlab/issues/1742#issuecomment-280949184, or mute the thread https://github.com/notifications/unsubscribe-auth/AABr0MZ6wkhfWucoMickhrcXPj84Avz1ks5reK9QgaJpZM4MFWEu .

-- Brian E. Granger Associate Professor of Physics and Data Science Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger@calpoly.edu and ellisonbg@gmail.com

[ddavidebor]

A few concepts. The first is my favourite

2nd and 3rd favourite

[stsievert]

How 'bout sticky notes?

[stsievert]

And we should probably indicate the In [] cells are draggable too, but in a different sense.

The arrows try to convey they're draggable, but only within this page.

[stsievert]

I've done a little more thinking; why do we need the boxes or different icons to encode as draggable?

[sccolbert]

I think tabs being draggable is obvious without an indicator, plus it frees up that icon for something more semantic for that tab.

[sccolbert]

We could also change the cursor when hovering over the input/output labels to the arrow-cross cursor.

[stsievert]

change the cursor when hovering ... arrow-cross cursor.

A mockup for clarity (generated before I realized above meaning):

I think we still need some other draggable encoding for the output. I'm not sure the in/out draggable encodings should be the same, or if the In draggable encoding should exist (besides the cursor changing on hover).

[ddavidebor]

Do you plan to add a similar function to the mirrored output for the input?

[stsievert]

similar function ... mirrored output

Can you clarify?

[ddavidebor]

A function to drag the input portion of the cell in a new tab, the same way you can drag the output portion of the cell into a new tab to obtain the mirrored output.

[stsievert]

That wasn't planned, but maybe that's not a bad idea.

On Feb 24, 2017, 6:15 AM -0600, Davide Bortolami notifications@github.com, wrote:

A function to drag the input portion of the cell in a new tab, the same way you can drag the output portion of the cell into a new tab to obtain the mirrored output.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/jupyterlab/jupyterlab/issues/1742#issuecomment-282277804), or mute the thread (https://github.com/notifications/unsubscribe-auth/ABQmG-u4LC1H42C3CWKe4XtSCJNaAQLeks5rfsnlgaJpZM4MFWEu).

[ddavidebor]

I was thinking about the possibility of doing something very symmetric with the drag zones:

• A: Drag to drag the whole cell inside the notebook's own tab
• B: Drag to drag input into a new cell tab
• C: Drag to drag output into a new cell tab

I think this solution, with nothing more than changing the pointer and the light grey background of the drag zone on hover, would be extremely clear.

[stsievert]

@ddavidebor nice solution.

B: Drag to drag input into a new cell tab

Could you run this cell when dragged to new tab/etc? What happens to any output?

[ddavidebor]

I don't see why not... control+enter could still be available. The output I think should remain in the old cell position, otherwise it would be like moving the whole cell outside the tab.

[stsievert]

The output ... should remain in the old cell position

That could be confusing, especially if the old output area can't be seen. If we provide some indication (either in output area or in the title) I think I'd be okay with this.

[ddavidebor]

The user could drag both input and output in new tabs (easily one on top of the other) in this scenario if they wanted.

What about an icon "tabs" like this only when the input/output has been dragged out into a tab? A click on the icon could trigger a focus on the tab.

Crappy mockup:

[stsievert]

A click on the icon could trigger a focus on the tab.

Hm... I'm don't know. I know I keep my personal workflow pretty minimal (few tabs etc). Does anyone else have opinions?

I like the changes proposed in https://github.com/jupyterlab/jupyterlab/issues/1742#issuecomment-282292947 and say we implement.

[ellisonbg]

@stsievert @ddavidebor I am not sure I understand the different drag zones in that comment. Can you clarify? The input prompt already allows dragging within a notebook and to other notebooks. I don't think we need different drag areas for those. I will try to spend some time on this stuff over the weekend though.

[stsievert]

@ellisonbg as I understand https://github.com/jupyterlab/jupyterlab/issues/1742#issuecomment-282292947:

• Zone A: current dragging input prompt functionality. Drag to reorder, drag to new notebook, etc.
• Zone B: mirror the input in a new tab/pane (new feature). This would provide parallel editing of different cells. This isn't possible right now (only one view of notebook allowed, have to scroll to see different cells)
• Zone C: current implementation (but change the cursor).

Example of what I envision for Zone B is below. Imagine the input prompt is selected and dragged (not the output prompt) and a new editing interface is visible after dropping.

For example, let's say right now I have to make changes in cell 4, but see the output in cell 5. I currently have to scroll up and down between edits. This new feature would provide a side-by-side editing.

[ddavidebor]

Yep as @stsievert wrote.

[ellisonbg]

OK I just caught up with the discussion. Thanks for helping with this stuff. Some thoughts:

• I agree that the discoverability of the cell and output dragging is low. Right now the input prompt shows the + arrow drag mouse pointer, but the output doesn't. Minimally, we should add that.
• I worry that the drag operations in the prompt area are already too complex. Right now, if you have a cell with N outputs, you have N+1 drag areas (input + N outputs). Because of that I think that adding yet another drag area would complicate things further.
• We also need to add UI for switch a single output to other MIME types. I am thinking of simple < > buttons that appear in the output prompt on hover that a user can click to rotate through the different MIME types. Keep in mind, this is different from a user switching between different outputs. Each of the N output can have M_N different MIME types.

To limit the complexity of the drag options, I would prefer to keep dragging (in the prompt areas) limited to the few, very obvious interactions and then expose other things using something like a small dropdown or context menu.

Also, please see my prompt explorations in #1691

Will post more ideas and mock ups...

[ddavidebor]

Ok.

Regarding the other discussion points, @ellisonbg what do you think about compacting the borders?

before/after mockup:

[ellisonbg]

@ddavidebor yes, getting rid of the double borders is one of the main things that needlessly complicates the visual look of our current UI. There are a couple of different state related things we will have to address, namely edit and command modes and code/markdown cells. Here is one possible idea:

Command mode of a code cell:

Edit mode of a code cell:

Edit mode of a Markdown cell:

[stsievert]

And we should probably trim the whitespace with this:

[ddavidebor]

Love it. How would the command/edit mode look for a cell with an output?

[ellisonbg]

Right now I have the left blue bar on the input alone, but it is easy to have it span the output as well. But it was a lot of blue ;-)

[ellisonbg]

Here is another with the blue fully to the L.

My only concern is that users will miss it because it is so far to the left?

[ellisonbg]

Here it is with a very subtle box shadow...

[ellisonbg]

Part of my motivation here is to keep the content area of the notebook looking as simple as possible. I find that with lots of shadows/solid fills/border, the notebook looks less and less like a document. I think having the blue cell "cursor" works well in this respect.

[blink1073]

I like the most recent one.

[ddavidebor]

My only concern is that users will miss it because it is so far to the left?

Kinda hard missing something that moves around :-)

Right now I have the left blue bar on the input alone, but it is easy to have it span the output as well. But it was a lot of blue ;-)

Maybe with the opacity down on the output half...

[blink1073]

The user would still have to find it to begin the initial interaction. I'm not sure about opacity, because we also have to encode multi-cell selection.

[ellisonbg]

Trying to play around with some animation we could trigger when the user enter in to edit mode:

https://jsfiddle.net/dmazc35w/3/

It works, but the problem is that a skew or rotation is by angle. For a large input, the rotation creates a large x offset that. To do this, we would need a fixed height element

[ddavidebor]

https://jsfiddle.net/j47h0L0j/3/ (I can't really write css XD sorry)

[ellisonbg]

We are also starting to work on design ideas related to real-time collaboration, where we will need the ability to distinguish between the cell you have selected/editing and the ones that others are working on. Here is a drawing where the current user is editing the first cell (blue) and other are editing the second one (orange):

I am also starting to play with the idea of using a outward protrusion on the R of the blue area when that cell is being edited to emphasize that state a bit more. Not shown here, but the other thing I am thinking about is to use a lighter grey for the prompt text of most cells and darken the selected cell slightly, again for emphasis. More mockups to follow...

[fperez]

I'm very happy to see lots of design exploration re. the UI, I just wanted to flag a couple of notes:

• I personally find the current (master) font mismatch between In and Out prompts very jarring and ugly. I'm not strongly married to a monospaced font there (they do take up more horizontal space, and were really only needed in a terminal to align input/output areas visually, which we can do with cell layout in the notebook). But I really think both of those areas ought to use the same font, whatever it is. For a while I thought it was just a bug, til I decided to search the tracker before reporting it :)

• Second, it's worth remembering that the In and Out names are also meant to convey that inputs and outputs are actually available to users as variables:

Now, we may decide this is an obscure enough feature (and it's IPython-specific) to sacrifice the visual cue, but there's a consistency there between UI and data that had a very good reason for existing.

In the context of long computations that one may have done and forgotten to assign to a variable, it can be very handy to 'recover' by typing x = Out[99] and then move on using x in an analysis. I did this all the time in Mathematica, which is why I transferred the capability over to IPython from day one. In fact, prompt output caching, with both Out and auto-generated _NNN variables were basically the motivation for creating IPython back in 2001.

So, not to say the feature is a "sacred cow" that can't go away, but I think it's important to note that it does exist for very clear reasons (pretty much all such details in IPython's origins were driven by concrete functional needs, never by capricious accident).

[ellisonbg]

Yes!

And there are even different levels of these APIs:

• Only one output per cell has the Out prompt. It is important to keep it in some manner so users know which output is tied into the variables you mention. This is universal to all kernels as this notion is baked into the message spec.
• The _NNN variables. Specific to IPython.
• The In/Out arrays. Specific to IPython.

My goal is to simplify as much as possible, without completely losing the visual expression of these APIs. We might decide to not express everything visually (could rename In/Out to I/O for example), but there needs to be some expression of these things.

On Wed, Mar 15, 2017 at 10:27 PM, Fernando Perez notifications@github.com wrote:

I'm very happy to see lots of design exploration re. the UI, I just wanted to flag a couple of notes:

-

I personally find the current (master) font mismatch between In and Out prompts very jarring and ugly. I'm not strongly married to a monospaced font there (they do take up more horizontal space, and were really only needed in a terminal to align input/output areas visually, which we can do with cell layout in the notebook). But I really think both of those areas ought to use the same font, whatever it is. For a while I thought it was just a bug, til I decided to search the tracker before reporting it :)

Second, it's worth remembering that the In and Out names are also meant to convey that inputs and outputs are actually available to users as variables:

[image: image] https://cloud.githubusercontent.com/assets/57394/23982920/fb2446f2-09cd-11e7-87c2-e71a3f1848c7.png

Now, we may decide this is an obscure enough feature (and it's IPython-specific) to sacrifice the visual cue, but there's a consistency there between UI and data that had a very good reason for existing.

In the context of long computations that one may have done and forgotten to assign to a variable, it can be very handy to 'recover' by typing x = Out[99] and then move on using x in an analysis. I did this all the time in Mathematica, which is why I transferred the capability over to IPython from day one. In fact, prompt output caching, with both Out and auto-generated _NNN variables were basically the motivation for creating IPython back in 2001.

So, not to say the feature is a "sacred cow" that can't go away, but I think it's important to note that it does exist for very clear reasons (pretty much all such details in IPython's origins were driven by concrete functional needs, never by capricious accident).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jupyterlab/jupyterlab/issues/1742#issuecomment-286961669, or mute the thread https://github.com/notifications/unsubscribe-auth/AABr0D0b4EdoMLgoEpMLpLaG2M2Y2bS3ks5rmMhcgaJpZM4MFWEu .

-- Brian E. Granger Associate Professor of Physics and Data Science Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger@calpoly.edu and ellisonbg@gmail.com

[stsievert]

@fperez hm interesting! I wasn't aware of that (though I was aware of "_n" availability at the IPython prompt).

How is the "In" array used, and how do you use it? I can see it being most useful to see a certain input from a certain cell/prompt, though I guess "x = eval(In[1])" could be used (though "x = Out[1]" would provide the same functionality).

[cameronoelsen]

Here is the same kinda concept that @ellisonbg showcased but includes the concept of quick actions to the notebook cell. Was talking with @eskirk and we played with the variation of a dropdown with extra actions additional to the run button.

Cell with quick actions (with more tab)

Cell with quick actions (with more tab expanded)

[fperez]

@stsievert to be honest, I don't think the In array ever sees much use these days... It was mostly put in for symmetry with Out, which is useful (though it overlaps with the _nnn variables).

As I mentioned, I think it's fine to evolve on these things and I don't mean to suggest that we should stay bound to all old decisions. Just providing context to how they came to be, so the context can inform future evolution for better results.

BTW, personally I really think the fonts for both in/out really should be the same. Especially if we make changes to shorten (say removing the words), it looks ugly to have them mismatched and misaligned for no particularly compelling reason.

I'm not sure I totally like the little >... in there, but I do love that you're trying to solve that problem :) It's tricky to find a nice visual solution for the extra actions without costing lots of vertical space, so maybe this is the best outcome for now. My mild -1 is only to encourage further exploration, not to shoot the idea down, b/c I do think it's worth doing something in this direction. Thanks!!

[fperez]

oops, misclick, sorry!

[eskirk]

I really like the @cameronoelsen mockup. Perhaps hovering over the blue indicator to the left of the cell could cause the mouse to change into the "click and drag" icon? I feel like that would be pretty intuitive.

[cameronoelsen]

Here is an initial concept I put together of another idea that we could run with. This feature would be used when someone is editing their notebook and they want to quickly create a new code or markdown cell in-between other cells in the document. I tried a multitude of different approaches, and it seemed that adding buttons and controls within the notebook document/in-between cells just cluttered the experience. I think this concept is really slick, but it necessitates animations to convey what is happening:

When a user places their cursor in-between two cells for a certain amount of time (a time amount that we will need to determine to find out how long of a hover is long enough to not be triggered accidentally all of the time), a pseudo cell appears between the two.

Questions I have for this approach:

• How annoying would it be, if at all, to the user to have this feature
• How long of a duration does the user need to be hovering over the area for the pseudo cell to appear
• Should the pseudo cell have the option between a code cell or markdown cell or should the pseudo cell, when clicked, just become a code cell similar to our current "new cell" action in the notebook toolbar at the top of the document