Proposal to manage Grafana chart as dependency

JorgeGar commented 1 year ago

As of now, Grafana is integrated as a deployment in the DevLake helm chart. I'd like to propose removing it and leveraging the official chart as a dependency. This would help keep up to date with Grafana's improvements and corrections, and it would standardise its usage.

klesh commented 1 year ago

Look good to me, @matrixji @IronCore864 what do you think?

JorgeGar commented 1 year ago

I guess the same could also be done for MySQL too.

IronCore864 commented 1 year ago

Yes, I agree that maybe we can use the official chart for mysql/grafana as dependencies.

hezyin commented 1 year ago

Good idea. Does Grafana's official chart support provisioning dashboards?

JorgeGar commented 1 year ago

Good idea. Does Grafana's official chart support provisioning dashboards?

Yes it does. On that sense it could be also interesting to see if it is possible to offer the dashboards to the user’s already existing Grafana.

IronCore864 commented 1 year ago

I will do some investigation on this one. Please assign this issue to me. @klesh

IronCore864 commented 1 year ago

I did some research, and I want to support this proposal.

Draft PR: https://github.com/apache/incubator-devlake-helm-chart/pull/140

Below are my detailed proposal; requesting for comments:

1 Background

The current situation:

DevLake supports both Docker compose, and Helm install
self-created Grafana Docker image with customized dashboard/datasource definition
Grafana Docker image releases together with DevLake, the Grafana version isn't the latest and can't be specified
DevLake Helm chart defines a customized Grafana Deployment/Service/Ingress/ConfigMap

2 Proposal

Disconnect DevLake with Grafana. By "disconnect", I mean:

remove the grafana folder from apache/incubator-devlake helm chart
move the customized dashboard/datasource definition to the apache/incubator-devlake-helm-chart repo
remove Grafana related K8s manifests from apache/incubator-devlake-helm-chart
use the official Grafana helm chart as a dependency for the DevLake helm chart

3 Pros and Cons

reduce DevLake Helm chart complexity
no reinventing wheels
customize Grafana version

klesh commented 1 year ago

Sounds good to me overall. However, we should also address the following problems:

The dashboards are highly dependent on the Domain Layer definition from the devlake repo, How do we keep track of them? e.g. dashboards developed for v0.18 might not work on v0.17. Keep in mind we have to make old dashboards should be upgraded or downgraded automatically while users upgrade/downgrad the helm chart. @IronCore864
Currently, we have some dashboards specifically designed for data verification (for both Testing and Debugging), should we deploy the helm in order to do so? if so, we should set up an automated CD system for testing. @IronCore864 @abeizn
How do we and users develop new dashboards or upgrade existing ones, we need documents to guide users on how to do so and submit their contribution. @IronCore864 @Startrekzky

klesh commented 1 year ago

How do we release, sop doc update

apache / incubator-devlake-helm-chart