search

apache / incubator-devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
https://devlake.apache.org/
Apache License 2.0
2.55k stars 508 forks source link

[Feature][UX] Validate token before running pipelines #8017

Open Startrekzky opened 1 week ago

Startrekzky commented 1 week ago

Search before asking

Use case

It took a user 40 min to 1 hour to find out a pipeline failed because there were invalid tokens.

It's much better to let the user know immediately after an expired token is detected.

Description

Use cases

No. use case
1 A project' has multiple connections (Jira & GitHub), and ALL of the tokens are invalid
2 A project' has multiple connections (Jira & GitHub), and only part of the token (e.g. Jira token) is invalid
3 A project' has multiple connections (Jira & GitHub), and one of the GitHub tokens is invalid

Expected result when triggering the pipeline in the above use cases

To make it simpler, all the above use cases will have the same result:

  1. When users trigger a pipeline in Collect data or Full refresh mode manually on the UI: a. prompt the user of invalid tokens in which connection(s). b. And, the pipeline will not run either.
  2. When users trigger a pipeline in Re-transform mode: The pipeline can be executed. This operation does not check the tokens.
  3. When a pipeline is automatically triggered (users cannot choose the sync mode), the pipeline fails immediately.

Tokens to be checked

All tokens of the connections under the project.

Prototypes

When there is one or more invalid tokens detected, prompt a dialogue to guide the user to update the tokan. image

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

d4x1 commented 1 week ago

Agreed with most of your advice. @Startrekzky

case 1, Config UI can use the existing API /plugin/connection/{id}/test, it will check the token. If necessary, I can create a new API to make Config UI simpler.

case 2,I'v made some progress which can be seen in pr https://github.com/apache/incubator-devlake/pull/7966, it still needs lots of work.

case 3, I want to make sure that: if there are two kinds of connections and the token of one of them are invalid, should the whole pipeline fail inmediatly? I am not sure about it. IMO, if github's token is invalid, then all github related tasks should failed inmediatly, but jira tasks should run correctly.

Startrekzky commented 1 week ago

case 3, I want to make sure that: if there are two kinds of connections and the token of one of them are invalid, should the whole pipeline fail immediately? I am not sure about it. IMO, if github's token is invalid, all github related tasks should fail immediately, but jira tasks should run correctly.

@d4x1 For case 3, I prefer this result When a pipeline is automatically triggered (users cannot choose the sync mode), the pipeline fails immediately for the following reasons:

  1. Around 80% of DevLake users measure DORA metrics, meaning only running Jira tasks, refdiff and dora tasks won't help users in this case. They will see the wrong DORA metrics after a partial-success pipeline.
  2. If the users want to fix the DORA metrics, they have to do another sync and wait, meaning that the time taken to collect the Jira data in point 1 will be wasted.
  3. The logic is simpler.