Config-UI Triggers Design Revamp

[e2corporation]

Describe the bug

Start revamp of Triggers page.

[hezyin]

@e2corporation I walked @Startrekzky @klesh and @mindlesscloud through your initial design today. @Startrekzky has some ideas and he will comment separately.

Please be aware that we have two design tasks: one is the interface for triggering data collection and two is the interface for showing the history of data collection pipelines. For the second part, we just introduced a new concept called Pipeline. It's essentially a group of Task in the same trigger request. Please read this #824 in details before you design this part. The design work itself for pipeline page is tracked in a separate issue, also assigned to you: https://github.com/merico-dev/lake/issues/835. We can use Gitlab's pipeline page as a useful reference.

[Startrekzky]

Hi @e2corporation , thanks for the design. I'll start with the users' goal in this scenario, which is to trigger data collection after data source configuration. Therefore, the design should:

• let users quickly realize that they should do the 'trigger collection' task
• provide an intuitive UI to tell users how to do the task
• enable users to complete the task as quickly as possible

Based on these goals, I have some comments about this design.

1. users have to manually 'toggle' on multiple data resources, this requires more clicks from users, while our goal is to reduce user clicking.
2. If users toggle on multiple data resources at the same time. Will the 'Enrichment Options' section automatically append? If yes, this will bring additional cognitive load to the users.
3. How about remove 'domain options' on this page, as the majority of users don't have to deal with it. We can keep it in the advanced mode.
4. The progress indicator should be consistent with the new pipeline for data collection #835. It'll be similar to something like below

[Startrekzky]

Here is my proposal for this page. You can take a look.

1. My idea is to show all form fields in this page by default, as there're not many, average 1 field for each plugin. If we don't do this and form fields are scattered on different 'status' of the page, it'll add more cognitive cost every time a user switches to another status (= toggles on a plugin)
2. We should add source_id field for Jira
3. See note 1-4 in the screenshots

[e2corporation]

@Startrekzky Thanks for your feedback, I've left responses to your questions below regarding the initial mockup attached on this ticket. At a high level I think this approach captures all the needed user goals you have outlined. Keep in mind that in today's version we have a page with raw json, which has to be manipulated before a user can run trigger. I do like your proposal sketch as an alternative layout idea, I'll address that in a separate comment.

users have to manually 'toggle' on multiple data resources, this requires more clicks from users, while our goal is to reduce user clicking.

I think you're overcomplicating the interpretation of this design a bit. Clicks are easy to accomplish on the one hand, it's very low effort for a user to tap on a switch. Our UI is already minimal style and this is no more difficult to operate than the existing interface. The reasoning behind the switches is that at page load, we can determine what data providers the user has configured and enable/disable the trigger option accordingly. This way we avoid sending a JSON request to the backend that won't work. To make things easier from your use-case, we can simply turn ON collection by default for all the integrations we have detected as "configured" and has a passing connection test. Additionally, if we know the user has not configured a provider and they attempt to toggle the switch on, we can force it back to off due to the logic checks.

If users toggle on multiple data resources at the same time. Will the 'Enrichment Options' section automatically append? If yes, this will bring additional cognitive load to the users.

The intent here with this approach is that the settings only show when a provider tile is selected. The instructions in the headline instruct the user to select a provider (by clicking the icon) and the enrichment options will switch to that provider. This way we don't have to show a bunch of settings at once.

How about remove 'domain options' on this page, as the majority of users don't have to deal with it. We can keep it in the advanced mode.

I agree this can be hidden in default mode all together and it can be utilized in the Advanced / RAW JSON mode.

The progress indicator should be consistent with the new pipeline for data collection Config UI design a pipeline page that's compliant with the new /pipelines api #835. It'll be similar to something like below

There screenshot you attached is just showing a console log of the docker image being built, so it's not entirely relevant to capturing the status of a task and showing % complete to the user. If you like the "terminal-style" look I can definitely work that into the UI at some level, but instead of overcomplicating this page all the progress needs to capture is the percentage complete of the pipeline, which in turn would be based on the tasks associated with that pipeline. Status text would be provided by the API if necessary and it would get displayed with the UI as polling is done to check status.

[e2corporation]

Here is my proposal for this page. You can take a look.

1. My idea is to show all form fields in this page by default, as there're not many, average 1 field for each plugin. If we don't do this and form fields are scattered on different 'status' of the page, it'll add more cognitive cost every time a user switches to another status (= toggles on a plugin)
2. We should add source_id field for Jira
3. See note 1-4 in the screenshots

I like this potential concept as a "Switchboard" type layout where we show all integrations stacked, I'll also work on a mockup with this approach.

[kevin-kline]

Just wanted to add my 2 cents in from today's discussion. The main idea I wanted to convey was I think we are lacking a visual representation for the User to create a pipeline. A pipeline is composed of one or many plugins, each which have their own configuration. Here is a very simple outline of the most basic idea I'm trying to convey. I hope this helps clarify what we want. If this is not helpful, please ignore. Just my thoughts. @joncodo @hezyin @e2corporation

[e2corporation]

@kevin-kline Thanks for the feedback Kevin, I think I understand your reasoning and I think we can clear up a few things on our next team discussion regarding triggers.

With the creation of the pipelines structure, Config-UI is now partly responsible for managing the pipelines in that we create named pipelines with a new POST request to /pipelines. This gives us the ability to have dedicated pipelines for GITLAB and JIRA etc, and if the user runs Trigger ALL we create another pipeline that's named "ALL PROVIDERS" (or something similar) with possibly a Timestamp appended.

I think the addition of an Add Button could be useful, but I think the approach we are taking is that the Pipelines should be an effect of user behavior, not something the user has to actively work on "building" or "adding" to the pipeline. So from that perspective the user doesn't need to know they have to build a pipeline, just that one or more exists based on how they operate the triggers page.

The triggers page will also instruct the user or provide link to visit the pipelines page for an overview of all the current running pipelines.

[Startrekzky]

@e2corporation Thanks for your reply. I want to clarify points 1, 2, 4

For points 1&2: I got your idea of 'not showing a bunch of settings at once'. If this is a solution for mobile device, I'm with you. Because mobile devices have small screen size, which makes it easier for users to complete tasks broken down to atomic actions in multiple steps.

However, in our case it's not neccessary. Why?

1. our config-UI is for mac/pc devices with big screen and form fields in this page are not many if we can a. only display the settings of the configured integrations. b. segment settings for different integrations to make it visually clearer

2. 'not showing a bunch of settings at once' interaction's side effect is 'NOT successfully configuring all settings' because: a. users may not realize that they should switch integration continue configuration after they've finished Jira setting, they might click the 'Trigger Collection' button below as it is the CTA of the page. b. users may forget to fill in some fields as once they've switched to another integration, the other settings are hide. c. users may input wrong values and it's hard to identify where the problem lies.

I have to further clarify, the preceeding reasons don't mean that we can't use the approach of 'not showing a bunch of settings at once' at all, but we should use it with some prerequisites.

1. The trigger page should have a clear flow/steps for users to understand: a. how many integration settings should be accomplished. b. only after all integration settings are accomplished, should they click 'Trigger Collection'. (Note: text hint is an option but the least effective solution, the better way is always 'visual guidance'.)
2. When users trigger collection without accomplishing all settings of configured, they should be notified.
3. Clear interation methods. I have some concerns about the current interaction plan as there's only 1 prototype. Can you upload more prototype to explain the interaction methods in the following scenarios? a. When users first entered this page with 'Jira', 'Gitlab', 'Jenkins' configured, what does this page look like? b. If users finshed configuring Jira and wanted to switch to Gitlab, what should he/she do? Which UI element shall he/she click? c. When users are configuring Gitlab, and they have already configured Jira. What does the page look like when they toggle off Jira.

For point 4: Sorry that the terminal screenshot is so confusing. I didn't want the progress indicator in Config UI the same look as it is in the terminal. My intent is to explain the progress indicator according to the new pipeline design may be more like step-based progress rather than percentage-based progress.

Step 1/9: Collecting Commits (150 so far...) or Step 3/9: Collecting Issues (1500 so far...) or Step 3/9: Collecting Issues (1500 so far...) requesting issue 1263 from api

You can see more details here https://github.com/merico-dev/lake/issues/822. And I also agree this is a task for next iteration.

[e2corporation]

Linked to #877