[SIP-67]Gallery modal redesign and enhancement

Note: This is 1/3 of a greater proposed effort in Data Visualization Enhancement, Reorganization and Revamp

Background

As of today, Superset has grown from a Hackathon project to a full-fledged BI tool with over 50 out-of-the-box visualization plugins offered. Users appreciate Superset for providing a variety of visualization options, though the organic growth of the product throughout the years has led to increasing difficulty in navigating in the gallery page, and creating visualizations efficiently.

Goal Posting 3 SIPs in related areas, and the shared end goal of SIPs 65, 66 and 67 is to improve the time-to-value in creating meaningful charts using Superset. |Less disorganized and stand alone plug-ins| Cleaner Gallery modal and control layout | More powerful charts!

Problem Statement

Problem 1- Disorienting Gallery page The visual clutter and design debt we accumulated the last few years makes it hard—especially for the new Superset users—to get a good idea of all the visualizations available at a glance, and to easily select the one that best serves their charting purposes.

Competitive analysis We did some competitive analysis to understand what are the main chart types other data analytics tools offer, and how others organize. In summary of the research, Superset offers a wider range of selections, compared to tools that are at similar maturity level, for example Mode, Metabase, Data Studio, Sisense, Logi Analytics and even Chartio.

Superset also has great advantages in extensibility and customization, thanks to the effort made in Dynamic Viz plug-ins system.

However, we identified some gaps compared to more mature BI tools like PowerBI and Tableau. Charts that we are missing in Superset include Combo/Composed charts, Project management charts(mentioned in roadmap item #), e.g. gantt, funnel and waterfall chart, and chart variations, e.g. column chart(horizontal bar), multi row big number/cards etc.

Based on the research we’ve done, there’s no question that integrating new charts that are mentioned above should also be part of the data visualization reorganization/redesign effort, in order to advance in the BI landscape.

Proposed Solutions for Problem 1 Gallery Page Redesign

We are taking a three-step approach to solve this problem. Step 1 : Categorize the existing charts

Option 1: Based on dataset type, Current category: Time-series, Comparison, Composition, Geospatial etc. https://docs.preset.io/docs/the-right-chart-for-your-data Option 2: Based on business use Popular, KPI, Analysis, Project management, Maps and Others Option 3: Abstract - D3 category Distribution, Ranking, Correlation, Part of a Whole, Evolution, Flow and Maps https://www.d3-graph-gallery.com/

Please note that whichever solution we land on, users may find the same chart appears in more than one chart categories.

Given the results of this poll, and incorporating other feedback gathered from the community by various channels, this SIP proposes to option 3

Step 2 : Decide which charts are “gallery worthy” Different BI tools have different sets of charts available in their galleries. We have to make this choice too, since we have several charts with several different variations, and layers of variations. For example, should “Pie Chart” and “Donut Chart” be 2 separate charts in the gallery? On one hand we can argue that they are essentially the same chart with just a visual difference. On the other hand, both of these charts are popular and many BI tools have them as separate options in their galleries. What about Scatter Plot and Bubble Chart? Other examples: Line vs smoothed line. Horizontal bar vs vertical bar vs stacked bar vs 100% stacked bar (this is an example of layers of variations) Area chart with circle markers vs area chart with triangle markers vs area chart with square markers (this is an extreme example) So, what variations of charts will we show in the gallery? Where do we draw the line between “too granular” and “too general”?

How we’re attempting to approach this question:

Look at comparable products 2.Come up with a list of principles that might guide what is chart worthy?
- "do people generally think of this as a visualization type?"
- should we make “x” chart a separate option from “y” chart in the gallery if: a) it adds a new dimension of understanding? b) it changes the shape of the visualization? c) It takes at least n number of control changes to go from x to y? d) n of the above principles are met?

Please let us know which of above principles should we take into consideration.

Poll: 1) Horizontal and vertical Bar are the same chart type 2) Donut and Pie Chart are the same chart type 3)Line and smooth line are the same chart type

Given the results of this poll, and incorporating other feedback gathered from the community by various channels, this SIP proposes to the last option, which is to add entries in the viz gallery for discrete and common variations of charts e.g. donut and pie, horizontal and vertical bar, area/stream/line/bar

Other considerations: What we select as gallery worthy will inform how controls on the control panel will be organized. Ideally, we would be able to make non essential changes to charts in the same place. And if charts x and y are two separate charts displayed in the gallery and they serve different purposes, in Explore to go from x to y you’d need to pull up the gallery to change as a conscious decision.

Step 3: Implement a robust viz search that help user select the right viz type Under the assumption that we want to make as many chart types visible in the hierarchy of the category as possible without overwhelming the users, we will be implementing a few new features in the gallery to improve the time-to-select-chart.

Text description, including general purpose, use case and supported data type etc.
Tag. e.g. tag Spider for Radar Chart as a.k.a.
Thumbnail example.
Advanced search bar to search all metadata of each chart type

We want to include an “All charts” category for users to quickly search across different sections.

Thanks for brainstorming: @mihir174 @villebro @mistercrunch @rusackas 🙏

Issue As a new user to Superset, I have been a bit confused when selecting a visualization type.

Suggestions Be automagical! issue: It is a bit confusing to have to select between two (or more) types of line charts — especially as there are little information available about how to decide when having to chose about it. Plus one may not know in advance which one to use, or one may change their mind after having clicked. suggestion: Allow the user to simply select “line chart” as automagically use a regular line chart or a time-series line-chart, depending on whether the abscissa is of a Timestamp data type. Group chart types issue: The “Select a visualization type” tile is a list of all visualisations available. There are a lot of possibilities and one might not notice a given type of chart. suggestions: group charts per categories: Line charts, Bar charts, Maps, etc. group charts per “function”/usage: e.g. one KPI only (big number, big number with line, indicator, bullet), one KPI vs time, data distribution, geographical data, etc.

Additional comments (beyond the polling options):

Something I find really nice and effective in the D3 graph gallery is that all chart types have a visually similar thumbnail (rather than the quite heterogeneous screenshots we currently have). This is not only more visually appealing (to me, at least), but also helps underlying the specificity of a given chart (cf. line vs. area plot) — and thus makes easier for the user to decide which to choose.
As shown by the second poll, some charts are “similar”/inter changeable, in the sense that they are variation of the same principle so axes/aggregation don't really change from one to another. Reducing user choices to chart categories only (e.g. “part of a whole”, “evolution over time”, …) and letting the user refine the variant in the Explore panel might be a bit extreme and frustrating for the user (“I want an 'area chart', why can I only choose something that look like a line chart?”) — especially if chart variants can be detailed under a same section. But I believe we should always offer “the right to make mistakes” and lower the cost of changing one's mind. Thus, if I chose “Line chart” (and then enter the Explore panel), I should be able to change to area without having to start defining my chart all over again (just like I can stack/unstack series or switch to “contribution” for some chart types). Thus, the litmus test for defining a visualisation type group could be: “Can I seamlessly switch to any of the other variants in this group after I've entered the Explore mode?”

I think the classification of existing charts is also very important. Does superset have plans in this regard?

I think the classification of existing charts is also very important. Does superset have plans in this regard?

@cdmikechen SIP 65 is written to tackle this, and we welcome your thoughts over on that thread.

Additional comments (beyond the polling options):

all chart types have a visually similar thumbnail

Agreed on this point. Having a simplified, slightly abstracted thumbnail, with a consistent visual style, would be quite nice. I think/hope we can tackle this, though it might not make it into the first iteration of the effort.

The other issue with this, however, is that several plugins in the future may be contributed by the community, and we'll have to find some way to both provide a design system/language, and hold these contributors to a certain standard.

Thus, the litmus test for defining a visualisation type group could be: “Can I seamlessly switch to any of the other variants in this group after I've entered the Explore mode?”

This is an intriguing idea, and would be worth mentioning on SIP-65 which goes into categorization. However, this could be a LOT of work to implement (it's not true for almost any combination of charts today), and/or would lead to a proliferation of categories if this becomes a requirement. It's definitely a good idea to strive toward though, and it would be the case if we subdivide certain charts (e.g. Pie and Donut, or some of the various line/area/bar implementations of the Timeseries chart)

apache / superset