mvisonneau / gitlab-ci-pipelines-exporter

Prometheus / OpenMetrics exporter for GitLab CI pipelines insights
Apache License 2.0
1.29k stars 243 forks source link

Wildcards not works with Gitlab Public URL (gitlab.com) #543

Open javier-torres opened 2 years ago

javier-torres commented 2 years ago

Hello,

I am configure your service exporter but it isn´t works for me. I am working whit Gitlab public and trying scraping wildcards with this configuration:

gitlab:
  url: https://gitlab.com
  # You can also configure the token using --gitlab-token
  # or the  environment variable
  token: XXXXXXXXXXXXXXXXX
wildcards:
  - {}

This not working for me, I think that is for the Gitlab URL is public.

image

I try the same configuration with private Gitlab URL and it´s working correcly.

gitlab:
  url: https://gitlab.example.com
  # You can also configure the token using --gitlab-token
  # or the  environment variable
  token: XXXXXXXXXXXXXXXXX
wildcards:
  - {}

image

I need filter all my project beacause is very complex add all projectis individually. With this configuration in Gitlab public URL is working correcty, but I have more projects and it is possible to add all of them and even the new projects that will be created.

gitlab:
  url: https://gitlab.com
  # You can also configure the token using --gitlab-token
  # or the  environment variable
  token: XXXXXXXXXXXXXXXXX
projects:
  - name: projects/project1
  - name: projects/project2

image

I am thinking about this, and y propose if is posible scraping wildcards for gitlab groups thats envolves more project into this groups such as this:

gitlab:
  url: https://gitlab.com
  # You can also configure the token using --gitlab-token
  # or the  environment variable
  token: XXXXXXXXXXXXXXXXX
groups:
  - name: group1
  - name: group2

If exist any configurations that allow wildcards in Gitlab public URL share with me please. Reading all docs and examples I haven´t been able to do works.

Thank you soy much. Javier Torres

rodridevops commented 1 year ago

Hi. Is there any updates on this? I'm trying to read all the projects from specific groups. Is this not implemented yet or i'm missing something?

jorikseldeslachts commented 1 year ago

bump

odinsy commented 1 year ago

faced with same problem

fredricfoo commented 1 year ago

I got the same issue, but was able to get it working with this wildcard setting.

  wildcards:
  - owner:
      name: <NAME_OF_YOUR_GROUP>
      kind: group
      include_subgroups: true

Upon digging the code further, the name field will be passed as id to list group projects api. You probably be able to use groupID as well

josephGlez commented 1 year ago

has anyone gotten this to work with self hosted gitlab?? I have this as well but project count comes back as 0. (we have 4k projects under 380 groups)

gitlab:
url: https://gitlab.example.com
# You can also configure the token using --gitlab-token
# or the environment variable
token: XXXXXXXXXXXXXXXXX
wildcards:
- {}

Is the API scope for the token enough?

tewfik-ghariani commented 1 year ago

@josephGlez Yes it is working on a self-hosted gitlab instance

As shared before, you have to specify the wildcard settings while setting the "\<group-name>"

  wildcards:
      - owner:
         name: "<group-name>"
         kind: "group"
         include_subgroups: true

I am using "read_api, read_repository" as scope of the token but I am not sure if API is enough or not

You have to ensure that the group access token in use has to be associated to the group specified in the wildcard, or any parent group ( not sub-group ) so that it has visibility on the projects

josephGlez commented 1 year ago

@tewfik-ghariani thanks for the speedy reply!! im an admin so was hoping to just grab everything without specifying the group. i’ll update my token and if push comes to shove, i will hust have to list out our 300ish groups. was really hoping the {} would do the trick! thanks a lot!!

Destrocamil commented 10 months ago

Faced the same issue and @tewfik-ghariani answer did the trick. Thanks a lot!