Add walkthrough to new dataset UI

[janagombitova]

Context

Why do we have this issue? What are we trying to solve? 

We want our products to make it easy to work with data. This also means that it should be easy to work with the products and thus easy to get started with different product features. Having a way to explain what different buttons do and how they behave is a nice way to get started with the tool.

How will this benefit the users?

• easier pick up of Flumen
• less guessing when working with the data grid

How will this benefit Akvo? 

• streamline the communication - we communicate where it is relevant
• ensure the product is explained in a unified and consistent way - by the product itself

Opportunity 

Current status quo

Currently, the only way users learn about the product pieces (or changes to them) is by exploring and reading the support documentation. It is ok but there are more friendlier ways to introduce features and changes to users.

A while back we decided to use https://introjs.com/example/hello-world/index.html in Flow to walk users through new features.

The idea 

The idea is to do the same for Lumen. To use IntroJs and to walk the users through the dataset view. Why? With supporting RQGs we are changing the data grid quite a lot (view, navigation, how derived columns behave). So this is a good opportunity to explain the changes to existing users and explain the data grid to new users.

• the change should be part of the RQG feature flag.
• we should introduce the different parts of the grid and what to expect (for example with Transformations you create a new column and this will land at the end of the data group)

Data set view components

• dataset header (user defined)
• autosave update
• transfor action
• transformation log
• visualise action
• data groups
• column header data type (can be changed by clicking on it)
• column header actions

Next steps 

• [x] add support for IntroJs
• [x] define introduction points
• [x] define copy
• [x] Have Georgia improve copy (she has time this week for us)
• [x] perhaps end by pointing at the Help button: If you ever need help, this is where to go.i

[Kiarii]

I think with this for now we should prioritize one main item

• the communication around splitting a (flow form) dataset into data groups - to be shown when the user's dataset has more than one data group (else the messaging would be meaningless since there would be no sidebar/datagroup nav)

• The comms around Repeated data group and Derived col indication are secondary and relevant when those two elements are in view..

• There is a number of components on the data view screen but I dont think we need to put each behind a walkthrough - this would be an over communication (which is what the intro.js library does with their examples) - we can achieve the same using tooltips (incl renaming a dataset) e.g. https://www.figma.com/proto/XQtjy781ugZ8dmmaA9hWnO/Lumen-style-guide?node-id=68%3A185&viewport=2492%2C1170%2C1&scaling=min-zoom [link subject to changes or removal]

• In addition, since HELP is already a FAB - All capped, only component with color (Green) bg, appearing in all screens - we do not need to highlight it with a walkvia; It is prominent and intuitive by itself, if it was someplace else perhaps doing so would be in order

[janagombitova]

All great points @Kiarii, well taken. My main goal with this issue to communicate the new data groups change somehow in the UI. So let’s indeed prioritize this. What would your suggestion be?

Could you and @marvinkome define and inplement the tool tips based on the design? Could we add one for the repeated group icon too?

[Kiarii]

Tentative idea: https://www.figma.com/proto/XQtjy781ugZ8dmmaA9hWnO/Lumen-style-guide?node-id=124%3A0&viewport=4413%2C1012%2C1&scaling=min-zoom

[janagombitova]

The idea is great! To the point and all that we need.

So we have two tasks:

1. Implement the tooltips.
   • Can we rethink the description of Transformations? The one there now is Transform dataset. This not really what happens. The user creates/derives new columns based on the data. Any other ideas?
   • Example: Akvo Flow datasets are split into data groups. A data group is based on form question groups and submission metadata.
2. Implement the 2 step walkthrough
   • For the 1st step I would suggest to revise the text as not all datasets are split into data groups, only those coming from Flow. So we need to make this clear. And the metadata group is not for the form, but about each submission.
   • For the 2nd step I suggest to write is as "instruction" rather than as an explanation: Select a data group to see the data in the grid. or is this not good practice?
   • How about adding another sentence here saying: Derived columns will appear at the end of the data group. or something in that line to explain the change in the behaviour of the derived columns?

[Kiarii]

On item 1

• have agreed with @janagombitova to drop the Transform and Visualize tooltips for now - assuming that these are self explanatory to Salim/Petra/et al.
• I have rephrased Flow datasets are split into data groups to "... are presented by data groups" - this is more indicative of what we are doing while splitting is a sth else. Lets keep in mind that if we group derived cols we will refactor the walkthrough text since datagroups will actually apply to all datasets

Item 2

• (in the works) we have agreed to consider adding a comms for derived col in the derived col notification as this is more in context in contrast to doing it on the landing. We could start as follow, however we might need to keep the snackbar longer on the screen so the user has time to read it.

  

  [@janagombitova I've the feeling that this is a bit of monkey business since Salim & co can perform a transform that creates a derived col without being (and therefore not ending) in the parent group, in which case our messaging on where der cols go makes no sense, even worse if a der col has its parents in different data groups. This perhaps reinforces the notion of simply grouping der cols - I actually noticed this is what we are doing in lumen-test]

• the suggestion to express data group selection as an instruction is valid - but since we by default select a datagroup and show its data lets keep this as explanation..

[Kiarii]

as a side note, I still think user created cols shd be noted as derived cols rather than transformations; as we have seen transformations refer to every change in structure or appearance applied to the dataset even when such do not result to a new col.. Put another way, a dataset can consist of 100+ transformations without there being a single user created col

[janagombitova]

@Kiarii all valid notes. I think for the initial iteration, the derived columns will remain grouped in the Transformation group and we will only move the RQG derived columns into the group. The next step (once supporting RQGs is handled) will be to figure out what to do with the derived columns.

So for now, for this issue:

• walkthrough with two steps: highlighting the new data group side panel, highlighting the selected data group (meta data)
• the walkthrough should once Salim opens a Flow dataset for the 1st time
• tooltip for the RQG icon
• tooltip for the transformation log

[Kiarii]

From one of the test sessions, we realized that it would be valuable to mention that in the light of presenting a dataset by data groups, all data relations are maintained, to assure Salim and co. that these are not lost or broken.

Basically we would need to mention this in the first screen when we introduce the new layout, I am still trying to think how we might phrase this - suggestions are welcome..

[janagombitova]

Looking good @marvinkome 👍 well done.

Can you specify here what backend changes are needed to finish up this task?

[janagombitova]

Works as expected