search

mvisonneau / gitlab-ci-pipelines-exporter

Prometheus / OpenMetrics exporter for GitLab CI pipelines insights
Apache License 2.0
1.27k stars 240 forks source link

Get non user defined pipelines variables to metrics label #96

Open mvisonneau opened 4 years ago

mvisonneau commented 4 years ago

Hiya :wave: I've just started experimenting with gitlab-ci-pipelines-exporter and I'm also finding it really useful. Thanks @mvisonneau! :heart:

I think I have a similar use-case to @misaon and thought it might be helpful to share on this issue.

I'd like to instrument how long our main build/test/deploy pipeline takes on the master branch so we can optimize those durations. However, we have a couple of additional scheduled pipelines that also target the master branch (for example, scheduled nightly renovate jobs). These do something totally different to our build/test/deploy pipeline. Currently it's not possible to omit these from the gitlab_ci_pipeline_last_run_duration_seconds{ref="master"} metric.

GitLab does provide a CI_PIPELINE_SOURCE variable that can be used to identify whether the pipeline was triggered from a schedule. If we could add this variable as a label to the metric, we could filter out scheduled master pipeline durations, and focus on what we need to measure.

There are a lot of pipeline variables though, so I think it would make sense to whitelist which ones are sent as labels. Alternatively if a general purpose solution is hard to architect, maybe we could enumerate some common use-cases for this and just support those.

Originally posted by @nfm in https://github.com/mvisonneau/gitlab-ci-pipelines-exporter/issues/33#issuecomment-617101133

nfm commented 4 years ago

Thanks for spinning this issue out of #33 @mvisonneau :grin: I think I understand now what that issue was scoped to.

I've had a dig through the GitLab API docs, and it looks like it might not be possible to retrieve the predefined variables that are set on particular pipelines, or individual jobs within the pipeline. I also can't find a way in the API docs to tell what the pipeline source was. The only reference I can find in the API docs is to the pipeline variables, which seems to only fetch variables from pipelines that are triggered manually.

Unless anyone else is across the API in more detail, this unfortunately may not be possible to solve!

geCarlo commented 1 year ago

Hi there 👋 , huge fan of this amazing tool. Just poking on this issue - any idea if this enhancement will be included in a release anytime soon? I see that the task is still considered blocked, but curious if GitLab release 14.3 has allowed for pipeline filters like source to be exposed (as seen on the GitLab docs here here)

geCarlo commented 10 months ago

Just a follow up from my last comment: although it may not be possible to capture user-defined variables, this tool has been updated to be able to filter on source, which accomplishes a piece of this issue I believe! :)

Thank you,