committime exporter not connecting to internal gitea repo

[jkmattatall]

OpenShift version

4.13

Problem description

I'm trying to connect a locally hosted OpenShift Pelorus Operator to a locally hosted Gitea instance. I've followed the steps provided in the OpenShoft Web Console here. This has worked, and I am able to open the Grafana instance and view the GUI, which displays N/A on all but Deployment Frequency > Current interval > and it usually displays a small multiple of two here.

Here is what I add to the Pelorus Operator's YAML file, with details redacted (all redacted details are confirmed correct):

      - app_name: committime-exporter
        enabled: true
        env_from_secrets:
          - gitea-secret
        exporter_type: committime
        source_url: <https://this.is.my.internal.gitea.link.net/organization/repository.git>
        source_ref: main
        extraEnv:
          - name: GIT_PROVIDER
            value: gitea
          - name: LOG_LEVEL
            value: DEBUG

Here's the secret I created with details redacted (all redacted details are confirmed correct):

oc create secret generic gitea-secret -n pelorus //
--from-literal=GIT_USER=<username> //
--from-literal=GIT_TOKEN=<full access token>//
--from-literal=SERVER=<https://this.is.my.internal.gitea.link.net/organization/repository.git>

I've tried variations with API_TOKEN, GIT_TOKEN, TOKEN, PASSWORD, as well as API_USER and USER. All have the same output/problem.

When I try to connect the committime exporter to my internal Gitea repository, the committime-exporter-1-build pod has an "Init:Error" with the following debug log:

Defaulted container "docker-build" out of: docker-build, git-clone (init), manage-dockerfile (init)
Error from server (BadRequest): container "docker-build" in pod "committime-exporter-1-build" is waiting to start: PodInitializing

When I check the Build details on the committime-exporter-1, this is always the log:

Cloning "https://this.is.my.internal.gitea.link.net/organization/repository.git" ...
error: failed to fetch requested repository "https://this.is.my.internal.gitea.link.net/organization/repository.git" with provided credentials

I am confident my username and token are correct. What could be wrong here?

Steps to reproduce

1. Locally host OpenShift and Gitea (I understand this is not something one will reproduce).
2. Follow the steps here to install the Pelorus Operator on OpenShift.
3. Attempt to connect a committime exporter using the Pelorus Operator's YAML I provided in the description.

Current behavior

When I try to connect the committime exporter to my internal Gitea repository, the committime-exporter-1-build pod has an "Init:Error" with the following debug log:

Defaulted container "docker-build" out of: docker-build, git-clone (init), manage-dockerfile (init)
Error from server (BadRequest): container "docker-build" in pod "committime-exporter-1-build" is waiting to start: PodInitializing

When I check the Build details on the committime-exporter-1, this is always the log:

Cloning "https://this.is.my.internal.gitea.link.net/organization/repository.git" ...
error: failed to fetch requested repository "https://this.is.my.internal.gitea.link.net/organization/repository.git" with provided credentials

Expected behavior

I expect the Prometheus connection to Gitea to be successful and Grafana to display updates to the GUI. I expect no error logs with the committime exporter pods/builds.

Code of Conduct

• [X] I agree to follow Pelorus's Code of Conduct

[mateusoliveira43]

Hello @jkmattatall , thanks for using Pelorus :smile:

From your configurations 2 points call my attention

• source_url and source_ref are used for development, unless you want to deploy your own Pelorus (create a fork in your Gitea instance), I would unset those
• I think some configuration options are with wrong name. Here is the list for committime

Can you try using this yaml and this secret?

      - app_name: committime-exporter
        enabled: true
        env_from_secrets:
          - gitea-secret
        exporter_type: committime
        extraEnv:
          - name: GIT_PROVIDER
            value: gitea
          - name: LOG_LEVEL
            value: DEBUG

oc create secret generic gitea-secret -n pelorus \
--from-literal=API_USER=<username> \
--from-literal=TOKEN=<full access token> \
--from-literal=GIT_API=<https://this.is.my.internal.gitea.link.net/organization/repository.git>

[jkmattatall]

Hi @mateusoliveira43! Thanks so much for the swift response. This appears to have worked, I now have committime/deploytime exporter pods set up and running, however I don't seem to be able to see any changes in Grafana when commits are made in the main branch in the repository. In Grafana I see these under Pelorus:

• Software Delivery Performance, everything here says N/A
• Software Delivery Performance - By App, Deployment Frequency > Current interval displays 1 for a few minutes on start, and then returns to N/A, after some more time the Application selector at the top can no longer select committime/deploytime

How would I test that the exporters are properly connected and see changes in Grafana?

[mateusoliveira43]

I think is everything working then. Probably grafana filter was last 5 minutes when you saw, then after this time, there were no new commits to show (check now selecting a big interval, like last 2 days, the 1 commit should show there).

I suggest you read through this to understand the measures https://pelorus.readthedocs.io/en/latest/GettingStarted/QuickstartTutorial/ its an example using GitHub and GitHub issues, but you could modify the configuration options to test with Gitea

[jkmattatall]

Thank you for the assistance again! I'm a Pelorus/metrics newbie so forgive my questions. It seems as though with just the Pelorus OpenShift Operator and the link to Gitea that it is not displaying any metrics. To me it looks like committime/deploytime in the Quickstart Tutorial you linked requires the project to be deployed in a namespace, and doesn't specifically have a link to the GitHub repository (besides the repo being what is deployed).

In your first comment:

Can you try using this yaml and this secret?

      - app_name: committime-exporter
        enabled: true
        env_from_secrets:
          - gitea-secret
        exporter_type: committime
        extraEnv:
          - name: GIT_PROVIDER
            value: gitea
          - name: LOG_LEVEL
            value: DEBUG

oc create secret generic gitea-secret -n pelorus \
--from-literal=API_USER=<username> \
--from-literal=TOKEN=<full access token> \
--from-literal=GIT_API=<https://this.is.my.internal.gitea.link.net/organization/repository.git>

There seems to be a link to the Gitea repo itself and not a deployed instance. Is this supposed to be strictly a link to an undeployed Gitea repository and not a deployed instance? If so, are metrics supposed to be gotten using committime/deploytime from just changes to a repo branch? If not, why does giving the exporters the Gitea repo matter?

[jkmattatall]

@mateusoliveira43 Hi again! I've come to the conclusion that I likely need a Gitea link and a deployed instance of something in that link in OpenShift in order to display statistics in Pelorus. So now I have a very basic Gitea repository with a folder and a basic deployable nginx yaml in it. My issue is that it's still not displaying in Grafana, I can't select anything on the top left drop-down despite there being a deployed a running nginx pod in the nginx namespace, and no statistics are displayed (even if I select a very long time range on the top right).

I deploy the nginx yaml in the exact same way as the Quickstart Tutorial, using git clone and then oc create. Below is my nginx.yaml:

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: nginx
    app.kubernetes.io/name: nginx
  name: nginx
  namespace: nginx
spec:
  containers:
    - image: nginx
      name: nginx
      resources: {}
  source:
    type: Git
    git:
      uri: https://this.is.my.internal.gitea.link.net/organization/repository.git
      ref: main

Here's the exporters section of my Pelorus Operator yaml:

  exporters:
    global: {}
    instances:
      - app_name: deploytime-exporter
        enabled: true
        env_from_secrets:
          - gitea-secret
        exporter_type: deploytime
        extraEnv:
          - name: GIT_PROVIDER
            value: gitea
          - name: LOG_LEVEL
            value: DEBUG
          - name: NAMESPACES
            value: nginx
      - app_name: committime-exporter
        enabled: true
        env_from_secrets:
          - gitea-secret
        exporter_type: committime
        extraEnv:
          - name: GIT_PROVIDER
            value: gitea
          - name: LOG_LEVEL
            value: DEBUG
          - name: NAMESPACES
            value: nginx

My gitea-secret is what you recommended above with API_USER/TOKEN/GIT_API, and I've confirmed those are correct. Do you have any ideas why it may not be recognizing my nginx pod in the nginx namespace?

[mateusoliveira43]

sorry for the late response @jkmattatall , I been busy with other projects

Yeah, you need deploytime/committime to see the two first metrics

I think just Pod will not work to Pelorus to see it as deployment (@mpryc @etsauer @KevinMGranger am I right?)

I would suggest forking the sample application from quickstart tutorial in your Gitea instance

[jkmattatall]

@mateusoliveira43 no need to apologize, I appreciate any and all assistance and the free time you're giving to respond!

The big issue is that I cannot use the example demo application from the Quick Start guide since my internal system does not allow external GitHub links, as much as I may want to follow the guide, everything has to be entirely internal. Is there a way to create a very simple/small app (such as a base nginx app/pod) that has no external links which Pelorus can easily recognize?

I've also noticed that if I leave NAMESPACES unassigned, or assign it to my Pelorus namespace, that I am able to select the deployed exporters from the drop down on the top left shown here: I am confused as to what the discrepancy is between the Pelorus exporter pods and my nginx pod which makes the nginx pod not displayed here. I've tried adding a Git source as well as the same labels that are on the Pelorus exporter pods to my nginx pod, to no avail. The Git source is in my previous comment, and here are the labels that I've added to my nginx pod:

What does Pelorus use/need to detect a pod if not labels?

[mateusoliveira43]

I would suggest using our demo as well https://github.com/dora-metrics/pelorus/tree/master/demo , instead of quickstart app, but I think it is a little broken

I will try to see later today if I can modify quickstart tutorial yaml so you can use it or provide you a simple app that you can use to test

About your question: its just the way Pelorus watches things in Openshift. The way Pelorus deploys its exporters (with DeployConfig) is one of the ways that Pelorus understands a deploy

[mateusoliveira43]

Hello @jkmattatall

So to answer your questions about Pelorus detecting Pods in OpenShift

for deploytime exporter, it looks all running pods (in the namespaces you specified) that are created by a ReplicaSet or ReplicationController. Code reference: https://github.com/dora-metrics/pelorus/blob/master/exporters/provider_common/openshift.py#L16

So, for pelorus to "see" a deployment, you would need a Deployment, for example.

If you want to test just deploytime exporter, I think this application is the simplest https://github.com/openshift/oadp-operator/blob/master/tests/e2e/sample-applications/nginx/nginx-deployment.yaml (just need to change app: for the APP_LABEL you are using).

For committime, it looks for Builds. Code reference: https://github.com/dora-metrics/pelorus/blob/master/exporters/committime/collector_base.py#L195-L201

I would suggest the Quick Start guide application for testing with committime. The 2 differences you would have is here https://github.com/konveyor/mig-demo-apps/blob/master/apps/todolist-mongo-go/mongo-persistent.yaml#L144 you would put the Gitea link for your fork (I do not know if you can directly fork from GitHub to Gitea, but if not, just copy the files) and you would not have the GitHub webhook, you would need to recreate the application for a new commit. This you can do? Or there limitation of your environment?

Please tell me if this helps you

[filiy]

Hey @jkmattatall I'm thinking you can close this issue now? We figured out to get commit time metrics by using ImageStreams, Image Registry info, or webhooks.