Request to make the default scraping metrics configuration configurable via helm values

Hello, We are migrating from Prometheus to k8s-monitoring, and we would like to make a slight modification to the default scraping metrics configuration, which is currently explicitly defined and cannot be overridden.

For example: Default Kube API scrapping rule in helm template

// API Server 
discovery.relabel "apiserver" {
  targets = discovery.kubernetes.services.targets
  rule {
    source_labels = ["__meta_kubernetes_namespace"]
    regex = "default"
    action = "keep"
  }
  rule {
    source_labels = ["__meta_kubernetes_service_name"]
    regex = "kubernetes"
    action = "keep"
  }
  rule {
    source_labels = ["__meta_kubernetes_service_port_name"]
    regex = "https"
    action = "keep"
  }
....

prometheus.scrape "apiserver" {
  job_name   = "integrations/kubernetes/apiserver"
  targets    = discovery.relabel.apiserver.output
  scheme     = "https"
  scrape_interval = {{ .Values.metrics.apiserver.scrapeInterval | default .Values.metrics.scrapeInterval | quote }}
  bearer_token_file = "/var/run/secrets/kubernetes.io/serviceaccount/token"
  tls_config {
    insecure_skip_verify = true
  }

Allowing customization of the scraping metrics configuration via Helm values would be nice. In this snippet we would like to make the following changes:

discovery.relabel "apiserver" {
  targets = discovery.kubernetes.endpoints.targets

rule {
    source_labels = ["__meta_kubernetes_endpoint_port_name"]
    regex = "https"
    action = "keep"
  }

Allowing changes to the default job names and define them in the values, as many of our Grafana dashboards rely on job labels with the current job names. We are not planning to change these job names in dashboard configs. Something like :

prometheus.scrape "apiserver" {
  job_name   = {{ .Values.metrics.apiserver.jobName }}

We don't want to entirely redefine this default config via extraConfig helm value.

Could you please update config templates in the chart?

grafana / k8s-monitoring-helm

Request to make the default scraping metrics configuration configurable via helm values #894