apache / incubator-devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
https://devlake.apache.org/
Apache License 2.0
2.61k stars 530 forks source link

[Question][external Grafana] won't create dora dashboard. #8013

Closed da-head0 closed 2 weeks ago

da-head0 commented 2 months ago

Question

Hello. I'm trying to use external grafana on helm install, but after pipeline finished, dashboard won't showing in my external grafana.

  # dependency chart values
  grafana:
    enabled: false
    #if grafana enabled is false, then external url should be provided
    external:
      url: "https://grafana.dev.company.io"
    image:
      repository: devlake.docker.scarf.sh/apache/devlake-dashboard
      tag: v1.0.1-beta6
    deploymentStrategy:
      type: Recreate
    adminPassword: "password" -> wich is correct admin password
    grafana.ini:
      server:
        serve_from_subpath: "true"
        root_url: "%(protocol)s://%(domain)s/grafana"
    #the secret name should be as same as .Values.option.connectionSecretName
    envFromSecrets:
      - name: "devlake-mysql-auth"
    #keep grafana timezone same as other pods, which is set by .Values.commonEnvs.TZ
    env:
      TZ: "UTC"
    persistence:
      enabled: true
      size: 4Gi
    ingressServiceName: ""
    ingressServicePort: ""

Screenshots

If applicable, add screenshots to help explain.

image

Additional context

the loadbalancer connected to external grafana only accepts my office VPN and VPC nat IP, which I think devlake use(VPC NAT IP). or should I add private IP CIDR from VPC?

dosubot[bot] commented 2 months ago

Hello, @da-head0! I'm here to assist you with any bugs, questions, or contributions. Let's work together to solve this issue.

The dashboard won't show in your external Grafana because you need to set the grafana.enabled value to false and provide the external Grafana URL using the grafana.external.url parameter in your Helm install command. Here is the correct configuration:

helm repo add devlake https://apache.github.io/incubator-devlake-helm-chart
helm repo update
ENCRYPTION_SECRET=$(openssl rand -base64 2000 | tr -dc 'A-Z' | fold -w 128 | head -n 1)
helm install devlake devlake/devlake \
  --set grafana.enabled=false \
  --set grafana.external.url=https://grafana.dev.company.io \
  --set lake.encryptionSecret.secret=$ENCRYPTION_SECRET

Make sure to replace https://grafana.dev.company.io with the actual URL of your external Grafana instance. For more detailed instructions, you can refer to the Apache DevLake Helm Setup documentation.

Additionally, if you are using minikube, you might need to forward the port for Grafana:

kubectl port-forward service/devlake-grafana  30091:3000

Then you can visit Grafana by URL http://YOUR-NODE-IP:30091 [1][2][3].

To continue talking to Dosu, mention @dosu.

da-head0 commented 2 months ago

@dosu yes, I put grafana.enabled: false, grafana.external.url: "https://grafana.dev.company.io" -> which is correct url from my company domain. And I'm not installing helm by cli, I'm using ArgoCD and sync Chart.yaml and values.yaml from ArgoCD to EKS directly.

dosubot[bot] commented 2 months ago

The DORA dashboard might not be showing in your external Grafana when using ArgoCD to sync Chart.yaml and values.yaml to EKS because the required data sources and transformations might not be properly configured. Here are the key points to check:

  1. Data Sources: Ensure that the following data sources are correctly configured and available in your Grafana setup:

    • Deployments from Jenkins, GitLab CI, GitHub Action, webhook, etc.
    • Pull Requests from GitHub PRs, GitLab MRs, BitBucket PRs, Azure DevOps PRs, etc.
    • Incidents from Jira issues, GitHub issues, TAPD issues, PagerDuty Incidents, etc.
  2. Transformations: Ensure that the necessary transformations are defined, particularly for deployments and incidents, as mentioned in the dashboard's introduction panel:

    - Transformation Required: Define `deployments` and `incidents` in [data transformations](https://devlake.apache.org/docs/Configuration/Tutorial#step-3---add-transformations-optional) while configuring the blueprint of a project.
  3. Validation: You can validate/debug the dashboard with the DORA validation dashboard.

Additionally, you can use the DORADebug.json dashboard to help validate your setup. This dashboard is designed to validate the DORA dashboard and includes specific panels to check metrics like "Deployment Frequency". Ensure that your Chart.yaml and values.yaml files include the necessary configurations for these data sources and transformations [1][2].

To continue talking to Dosu, mention @dosu.

github-actions[bot] commented 3 weeks ago

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 2 weeks ago

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.