dora-metrics / pelorus

Automate the measurement of organizational behavior
https://pelorus.readthedocs.io/
Apache License 2.0
241 stars 82 forks source link

Open a GitHub Issue does not reflect the date in Change Failure Rate dashboard #1097

Closed pacopeng closed 7 months ago

pacopeng commented 8 months ago

OpenShift version

4.12

Problem description

after I create a issue in github and tag the issue with "bug" and "app.kubernetes.io/name=todolist" the data does not reflect in "Change Failure Rate" Graph. Once I closed the issue, the data reflect in the "Mean Time to Restore" graph.

Steps to reproduce

  1. install the Pelorus v0.0.7 through Operator hub.
  2. create pelorus instance with below yaml
    apiVersion: charts.pelorus.dora-metrics.io/v1alpha1
    kind: Pelorus
    metadata:
    annotations:
    operator-sdk/primary-resource: /pelorus-pelorus-thanos
    operator-sdk/primary-resource-type: ClusterRoleBinding.rbac.authorization.k8s.io
    resourceVersion: '1490913860'
    name: pelorus-sample
    uid: 8866abfc-3ab7-428c-81d9-86b0304f102a
    creationTimestamp: '2023-10-30T10:07:02Z'
    generation: 33
    namespace: pelorus
    finalizers:
    - helm.sdk.operatorframework.io/uninstall-release
    spec:
    prometheus_storage: true
    openshift_prometheus_basic_auth_pass: changeme
    prometheus_retention: 1y
    custom_ca: true
    openshift_prometheus_htpasswd_auth: 'internal:{SHA}+pvrmeQCmtWmYVOZ57uuITVghrM='
    exporters:
    instances:
      - app_name: deploytime-exporter
        enabled: true
        exporter_type: deploytime
        extraEnv:
          - name: LOG_LEVEL
            value: DEBUG
          - name: NAMESPACES
            value: mongo-persistent
      - app_name: failuretime-exporter
        enabled: true
        env_from_secrets:
          - github-secret
        exporter_type: failure
        extraEnv:
          - name: LOG_LEVEL
            value: DEBUG
          - name: PROVIDER
            value: github
          - name: PROJECTS
            value: pacopeng/mig-demo-apps
      - app_name: failure-exporter
        enabled: false
        exporter_type: failure
        extraEnv:
          - name: NAMESPACES
            value: mongo-persistent
          - name: PROVIDER
            value: github
          - name: LOG_LEVEL
            value: DEBUG
          - name: TOKEN
            value: >-
              github_pat_<token>
          - name: PROJECTS
            value: pacopeng/mig-demo-apps
          - name: GITHUB_ISSUE_LABEL
            value: bug
          - name: APP_LABEL
            value: app.kubernetes.io/name
          - name: TLS_VERIFY
            value: 'false'
      - app_name: committime-exporter
        enabled: true
        exporter_type: committime
        extraEnv:
          - name: LOG_LEVEL
            value: DEBUG
          - name: PROVIDE
            value: git
          - name: GIT_PROVIDER
            value: gitlab
          - name: API_USER
            value: root
          - name: NAMESPACES
            value: mongo-persistent
          - name: TOKEN
            value:< gitlab_token>
      - app_name: committime-exporter3
        enabled: true
        exporter_type: committime
        extraEnv:
          - name: LOG_LEVEL
            value: DEBUG
          - name: NAMESPACES
            value: mongo-persistent
          - name: PROVIDER
            value: image
          - name: COMMIT_DATE_ANNOTATION
            value: io.openshift.build.commit.date
      - app_name: committime-exporter4
        enabled: false
        exporter_type: committime
        extraEnv:
          - name: NAMESPACES
            value: mongo-persistent
          - name: PROVIDER
            value: containerimage
          - name: COMMIT_DATE_ANNOTATION
            value: io.openshift.build.commit.date
      - app_name: webhook-exporter
        enabled: true
        exporter_type: webhook
    prometheus_storage_pvc_capacity: 2Gi
    prometheus_retention_size: 1GB
    prometheus_storage_pvc_storageclass: ocs-storagecluster-ceph-rbd
  3. create issue label "app.kubernetes.io/name=todolist"
  4. create a issue and attach the label "bug" and "app.kubernetes.io/name=todolist"

Current behavior

  1. BUG1 issue created image
  2. no date reflect in "Change Failure Rate" image
  3. close issue BUG1 image
  4. the data reflect in "Mean Time to Restore" image

Expected behavior

when I create the BUG1 issue , the data should reflect in "Change Failure Rate"

Code of Conduct

pacopeng commented 8 months ago

logs for failure exporter ############ failuer-exporter.log

#############

mateusoliveira43 commented 8 months ago

Thanks for using Pelorus @pacopeng :smile:

I will try to investigate this today. Get back to you when I have some ideas

mateusoliveira43 commented 7 months ago

Hello @pacopeng

I think the problem you faced is that you had only failure exporter up

For "Change failure rate" you need both failure and deploytime exporters. Reference: https://pelorus.readthedocs.io/en/latest/philosophy/outcomes/SoftwareDeliveryPerformance/

Can you test if this fixes your problem?

pacopeng commented 7 months ago

@mateusoliveira43 Thanks for the clarification. I test it , it works . first I start a new deployment , once the data reflect in the Deployment frequency graph, the I create the issue immediately and then the Change Failure Rate data appeared. the configuration of deployment exporter was there, but I think the reason why Change Failure Rate not there before is the Pelorus only collect deployment time 30 minutes before. maybe I don't create the issue during the time period.

here is the screenshot of success

image
pacopeng commented 7 months ago

I closed this issue as it works. Thanks for your help.

mateusoliveira43 commented 7 months ago

Yes, Pelorus collects data that is max 30min years old, because of Prometheus that has this restriction, that is where Pelorus stores the data its collects