drag & drop next steps

[Janpot]

Visual editor

This RFC tries to specify a paradigm for our visual editor. For internal tooling we want something that allows for flexible layout, yet is constrained enough to require minimal manual tweaking.

Regular components

• All components conform to the width given to them by the layout algorithm. For some components this means they will have to centre themselves in a box (e.g. a button my take width from its content, which may be more narrow than the available width) some components will have an option to expand to the full width of their box (e.g. fullWidth property on TextField). Other components will horizontally adapt to any width imposed on them (e.g. image, datagrid)
• All components define the height they want to be rendered at. For some components this will be always the same, e.g. textinputs have a fixed height. And for some it will be configurable. e.g. data grid or image

Layout components

In terms of layout we identify two types of components, columns and rows.

column

a column works vertically. It acts as a component on the horizontal axis (adapt to the width imposed to it by the layout algorithm). It serves as the container for rows. Its only possible children are rows and its only possible parent is a row. A only needs to be manually created when the user wants to build something like a sidebar.

• A column only has rows as its children. Dropping a component on a column where there is no row, implicitly creates a row. Removing the last child from a row, removes that row from the page.

rows

Rows are never created explicitly, they are implicitly created depending on the drop position of certain elements

• A row has the same width as the column it’s under.
• A row is a 12 column grid layout into which its children take their place.
• A child of a row takes up a certain amount of columns (span). The user can adjust the span of an element with drag handles on the left and right side of the element.

page

On the top level, a page acts as a column

Drop algorithm

We must avoid layout shifting as much as possible, therefor we will not live preview elements as they are dragged on the page. Dragging a new element only highlights cursors as available drop targets. Dragging an existing element, to move it around, stays in place until the drop is performed.

The biggest challenge of drag and drop visual editor is to calculate a predictable insertion position for the drop target. I propose the following algorithm:

• around top level rows there is a 5px (?) band which always takes precedence as a drop target to create a new row that contains the element being dragged. We may shift the page rows apart by the same gap during dragging if this can be done with minimal layout shift.

For any other [X,Y] mouse position during the drag

• find the most deeply nested component under the mouse
• divide the component into 4 quadrants by connecting its corners diagonally, determine which quadrant the mouse is over to determine the side of the box that will be the drop target.
  • is it left or right: insert a new component to the left of the right within the same parent row as the one of the element that’s hovered (TODO: what if there’s no more space left in the row, a.k.a. all of its columns are occupied? Disable the drop at this location?)
  • is it top or bottom:
    • If the element is the only child of its row, add another row above or below that row with the dropped element as its only child
    • If the element has siblings in its row, replace it with a new column containing two rows, one with the original hovered element and one with the dropped element
• TODO: we need to special case moving around an element within its own row.

This algorithm takes care of the most common situations, top level inserts and inserting at the deepest level. In very deeply nested layouts it may be necessary to insert an element in between two nested layout components. I expect this to be an edge case for internal applications, but it can be solved by designating a keyboard modifier (shift or Cmd) that opens a context menu with all the container components in the same axis which can then be hovered to specify the exact insert position.

Resizing components

Should changing the width of a component or moving it around in its row be constrained by the available columns. Or should rows be allowed to overflow. Will overflow behavior be necessary to allow responsive design? (e.g. width = 3 columns when lg, 6 columns when sm,...)

Benchmark

This RFC borrows some ideas from makeswift.com. Stronlgy encouraged to familiarize yourself with its editor canvas implementation.

considerations

• make column count configurable? count, gap,...
• on the component level, we could mandate a minimum span?
• For elements that have configurable height we can add a drag handle at the bottom, but that doesn’t have to be part of this RFC
• For top level rows we may want to add an option to constrain the width, to create a fluid responsive container

[prakhargupta1]

As discussed with @Janpot closing this issue.

[Janpot]

Rewritten to describe a more complete visual editor paradigm.

[oliviertassinari]

In addition to a drag and drop UX, I think that a UX that would behave like Notion could feel really great (maybe?). As a developer, I feel like I'm wired to build apps by starting with finding on the page what component and where to add next. Having to go to the side to find a component to drag it feels slower than being able to stay focused on the area of interest. So it could feel closer to pro-code: type code to add it where you want it to be. The workflow could be as follow:

1. Hover the element next to the new one you want to add.
2. A + icon button is shown.

https://user-images.githubusercontent.com/3165635/168452784-d9fd3f68-019a-456c-a702-dbe6a4dd34e2.mp4

4. Press the button, a combobox is displayed

6. Type in the list the component you want to add
7. Press Enter to add the current highlighted component in the combobox.

[mbrookes]

The workflow could be as follow[s]

Not having to navigate to and browse the component panel would also lend itself to a nicer keyboard interaction model for power-users/accessibility.

[prakhargupta1]

I guess https://github.com/mui/mui-toolpad/issues/359 will be a part of this enhancement?

[apedroferreira]

Here are some ideas for an initial grid system we could use, fully based on the Material Design guidelines. https://material.io/design/layout/understanding-layout.html#principles

I think it looks like a great starting point to reason about the drag & drop system to implement:

The responsive layout grid is made up of three elements: columns, gutters, and margins.

1. Columns
2. Gutters
3. Margins

Columns

Content is placed in the areas of the screen that contain columns. In responsive layouts, column width is defined with percentages, rather than fixed values. The number of columns displayed in the grid is determined by the breakpoint range, a range of predetermined screen sizes. A breakpoint can correspond with mobile, tablet, or other screen type.

Gutters

A gutter is the space between columns that helps separate content. Gutter widths are fixed values at each breakpoint range. To better adapt to a given screen size, gutter widths can change at different breakpoints.

Margins

Margins are the space between content and the left and right edges of the screen. Margin widths are defined using fixed or scaling values at each breakpoint range. To better adapt to the screen, the margin width can change at different breakpoints. Wider margins are more appropriate for larger screens, as they create more whitespace around the perimeter of content.

Breakpoints

A breakpoint is the screen size threshold determined by specific layout requirements. At a given breakpoint range, the layout adjusts to suit the screen size and orientation. Material Design provides responsive layouts based on 4-column, 8-column, and 12-column grids, available for use across different screens, devices, and orientations. Each breakpoint range determines the number of columns, and recommended margins and gutters for each display size.

Recommended values for screen sizes/margins/number of columns here: https://material.io/design/layout/responsive-layout-grid.html#breakpoints

Layout Spacing

I suggest that we default to a grid with the same spacing as in the Material UI guidelines:

• align major elements such as container to an 8px grid (mobile, tablet and desktop)
• align minor elements such as icons and text to a 4px grid
• exception: text (and other elements too, probably?) can be placed outside of the 4px of a grid when it’s centered within a component

Element Spacing and Sizing

Every element should exist within a rectangular container. Containers can be rigid and restrict the size or crop of elements within them, or they can be flexible and grow to support the size of the content they hold.

The following properties should be adjustable within an element:

• space between elements within the component (padding)
• dimensions (width and height) of elements within the component
  • height should be specified in multiples of 8px, while width should be responsive according to viewport width
  • certain elements might have specific constraints for dimensions, margins or paddings, such as for example maximum width
• placement of elements within the component (alignment)

When scaling a layout, components can have fixed or responsive widths within the range of size constraints. Elements with fixed widths remain the same width regardless of a screen’s size. Elements with responsive widths expand and contract as a screen size changes.

[apedroferreira]

Layout Composition

A large screen layout has three main regions:

1. App bars
2. Navigation
3. Body

Body region

The body region is used for displaying most of the content in an app. The body region uses scaling values for three parameters:

• Vertical and horizontal dimensions
• Number of columns
• Margins

Common Dashboard Design Patterns

Here are some common design patterns for dashboards / internal tools that we might want to accommodate for.
 This research was based on competition examples and case studies, templates from similar tools and dashboard UI designs found around the web. I believe the listed items would be able to cover every single aspect of these designs.

• Container headers or footers with left, center or right-aligned elements or groups of elements, or full-width elements such as an horizontal tabbed area

• Containers with nested containers that take up different heights or numbers of columns

• Left, center, right-aligned or full-width elements or groups of elements such as filters, search bars, action buttons and text - either in nested containers or the root page container

• Scrollable tables

• Horizontally evenly-spaced elements inside a container

  

• Full-height, fixed-position sidebars with horizontal nav items, section headers or any other elements or element groups (could be vertically scrollable)

• Full-width or partial-width page headers or footers with left or right-aligned elements or groups of elements, and/or full-width elements that take up the remaining space

• Horizontally and vertically divisible grid layouts

• Containers with customizable spacing between their components (regardless of grid gutter spacing)

• Full-width sections, tables, forms and charts

• Sequences of centered horizontal elements or element groups

• Hero sections with image on one side and heading + description + action buttons on the other side
• Multi-column sections with centered elements such as images and text

• Collapsible sections inside columns
• Multi-level vertical item lists (with collapsible items)
• Left, right-aligned or justified forms
• Multi-column layouts

[apedroferreira]

Based on the RFC above, and after looking at many competing tool UXs, here's how I think the next version of our visual editor could work. I think the following would allow us to create any design above and would be very similar to @Janpot's suggestion, with maybe just a few differences.

Regular components

I agree with everything Jan wrote in this section, and it all seems to be according / easy to conform to something like the Material UI design guidelines.

Layout components

There would be 2 types of layout components: rows and columns. There would be one single page layout grid with a certain number of columns. All layout component widths would span an integer number of columns of the page layout grid.

We could have an explicit column component, as Jan defined it. A column’s width would be defined relative to its parent column as a fraction of its dimension, always faithful to the grid units (so it would always be an integer number of columns, and the minimum width would be 1 column). A column’s height would be defined by the height of its content. Only a column's width could be changed with drag handlers.

The top-level column (the page itself) would be a full-width column, and its minimum height would be the height of the screen/viewport.

A column could only contain rows, and components (including columns, except the top-level column) could only be placed inside rows. Anything added directly inside a column (even another column) would automatically create a row to contain it. Rows could or could not be created explicitly, I think this could work either way. A row would have the same width as the column it’s inside.

 Only a row's height would be resizeable with drag handlers, not its width.

Rows, just like columns, would be configurable components with the ability to change properties such as their background color, margins and padding. However, a row/column's total width, including margin, should always fit exactly in the grid columns.

I don’t think a non-layout element’s horizontal span would need to be adjusted in relation to the grid, but in relation to its parent row (or other type of container component) only.

For responsive layouts, the largest grid would have 12 columns, and then for smaller screens we could have one with 8 columns and the smallest one with 4. I don't think we should use any grid gutter as it just complicates things. We would need to find some way for components to be able change between these responsive layouts and still look good, but that should probably be a separate feature.

Drop algorithm

I think it’s a good idea not to have any layout previews for now, and I agree that they can be confusing. 
We can just highlight certain parts of the UI in green or red for example, whether the component being dragged is droppable there or not. 
 
We can try to show highlight bands without causing any layout shift, by having them overlap what’s in the page instead of adding new space.

 
 
I would say the drag and drop could work something like this:

• 
Dragging to the sides of a component - place the new component before or after the component, in the same row. If there is no more space in the row, we could have it overflow and look bad, we could hide the overflow and make it scrollable, we could prevent the user from placing… I guess we can look into solutions for this separately after the new drag and drop is implemented.
• Dragging to the top or bottom of a component - place the new component in a new row above or below, inside the same column the component is in.
• Dragging to the center of a component that can work as a container - place the new component inside the container component.

I like the idea of splitting each component into quadrants with diagonal lines - we just might need to reserve some area in the center of container components too.

I agree that the most deeply nested element should always be the one selected if there’s multiple elements under the cursor.
 I like the idea of having some kind of modifier or hover menu for selecting components at multiple levels. A map view of the component structure would be very helpful too, but that should probably be a separate feature.

Notion-style UX

I believe we could add something like this later on top of this drag and drop system, so different users could have different options? I feel like dragging and dropping is the most intuitive way to build a web page interactively.

---

I'm sure there would be flaws and unpredicted obstacles in this concept, but what we need is a concept good enough for me to start experimenting with and we could make changes along the way, and try fixing any flaws that come up. Feel free to discuss this and send feedback!

[bharatkashyap]

Closed by https://github.com/mui/mui-toolpad/pull/466 @apedroferreira?

[apedroferreira]

Closed by #466 @apedroferreira?

i'm working on this https://github.com/mui/mui-toolpad/issues/359, which could be kind of considered a sub-issue of this one. so maybe let's close both once we have the resizing?

[apedroferreira]

Closing this issue as basic functionality for everything mentioned here seems done to me, even though there's lots of room to improve these features of course.