search

apache / incubator-devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
https://devlake.apache.org/
Apache License 2.0
2.56k stars 510 forks source link

[Bug][DORA] Webhook Deployment not appearing in DORA metrics dashboard #8059

Open vincentpatella opened 1 day ago

vincentpatella commented 1 day ago

Search before asking

What happened

I set up a project with only a webhook as the data input and made an api call on my local machine to generate a deployment. The API call returns a success code, and I see the deployment when I look at the lake.cicd_deployments and lake.cicd_deployments_commits tables, but I don't see a mapping for it in the lake.cicd_scopes table, which I think is the main issue. Because there's no mapping for lake.cicd_scopes (there are mappings for my jenkins and AzureDevOps deployment pipelines), it doesn't show up in the DORA metrics dashboard. I followed the steps here to set it up: https://devlake.apache.org/docs/Configuration/webhook/

What do you expect to happen

I expect the deployment to show up on the DORA metrics dashboard like it does for my Jenkins and Azure DevOps deployments.

How to reproduce

  1. Create a new project named Webhook Test

  2. Create a new webhook and name it Webhook Test

  3. Run Collect Data

  4. Create an API key to authenticate with DevLake

  5. Make the following API call (using your own url):

    curl http://localhost:4000/api/rest/plugins/webhook/9/deployments -X 'POST' -H 'Authorization: Bearer {MY_API_KEY}' -d '{ "id": "Test 2", "createdDate":"2024-09-02T11:00:00+00:00", "startedDate":"2024-09-02T12:00:00+00:00", "finishedDate":"2024-09-03T13:00:00+00:00", "environment":"PRODUCTION", "result": "SUCCESS", "displayTitle":"optional-custom-deploy-display-title", "name": "Test Name", "deploymentCommits":[ { "repoUrl":"required-repo-url", "refName": "optional-release-v0.17", "startedDate":"2024-01-01T11:00:00+00:00", "finishedDate":"2024-01-02T11:00:00+00:00", "commitSha":"c1", "commitMsg":"optional-msg-1", "result":"SUCCESS", "name":"Test Commit 1", "displayTitle":"Test Commit 1" }, { "repoUrl":"repo-2", "refName": "optional-release-v0.17", "startedDate":"2024-01-01T11:00:00+00:00", "finishedDate":"2024-01-02T11:00:00+00:00", "commitSha":"c2", "commitMsg":"optional-msg-2", "result":"FAILURE", "name":"Test Commit 2", "displayTitle":"Test Commit 2" } ] }'

  6. Run Collect Data

  7. Check the DORA or DORA Details - Deployment Frequency dashboard

Anything else

I have tried multiple projects and multiple webhooks, but still have this problem with each one.

Version

v1.0.1-beta8@e195107

Are you willing to submit PR?

Code of Conduct

dosubot[bot] commented 1 day ago

Hello @vincentpatella! 👋 I'm here to help you with any bugs, questions, or contributions you have for the project. Let's tackle this together!

I found a similar closed issue: [Bug][DORA] Webhook incidents do not appear, which was resolved by a pull request: fix: add webhook boards in project_mapping table on August 16, 2023 [1].

The deployment created via webhook is not appearing in the DORA metrics dashboard because it is likely not associated with a DevLake project. To resolve this, follow these steps:

  1. Go to the Projects page in DevLake.
  2. Create or select the relevant project.
  3. Add the existing webhook to this project.
  4. Ensure that the data collection is started by clicking Collect All in the blueprint status tab.

This association is necessary for the deployment data to be included in the DORA metrics dashboard. Additionally, ensure that the webhook configuration is correctly set up to push deployment data to DevLake, as described in the documentation. This includes verifying that the webhook payload schema matches the expected format and that the data is being correctly recorded in the cicd_deployments and cicd_deployment_commits tables [2].

For more detailed guidance on setting up and troubleshooting DORA metrics, you can refer to the DORA implementation guide and the DORA troubleshooting documentation. Additionally, you can review the DORA metrics documentation for comprehensive information on implementing and configuring DORA metrics in DevLake [3][4][5].

Furthermore, if you are using webhooks to push deployment data, ensure that the payload schema is correctly formatted. The required fields for deployment data include id, startedDate, finishedDate, and deploymentCommits with their respective repoUrl, commitSha, startedDate, and finishedDate. For more details on the payload schema and sample API calls, refer to the Webhook Plugin documentation [6].

To continue talking to Dosu, mention @dosu.

vincentpatella commented 1 day ago

Hi, I had already looked at that issue and it did not help. The webhook is also already associated with a project and I copied the payload from the Webhook documentation for the fields before editing the data to match what a typical deployment would expect.

Startrekzky commented 1 day ago

Hi @vincentpatella , we'll look into this problem.