open-telemetry / opentelemetry-operator

Kubernetes Operator for OpenTelemetry Collector
Apache License 2.0
1.13k stars 396 forks source link

Ability to disable Prometheus scrape annotations #2554

Open GiorgioBullo opened 5 months ago

GiorgioBullo commented 5 months ago

Component(s)

Collector

Is your feature request related to a problem? Please describe.

When creating an opentelemetry collector, I noticed that the operator always creates the deployments with the following annotations:

    prometheus.io/path: /metrics
    prometheus.io/port: '8888'
    prometheus.io/scrape: 'true' 

I had a look at the code, and it looked to me like they are hardcoded in internal/manifests/collector/annotations.go

In my environment, I am using a servicemonitor to scrape the collector, so I don't want to have those annotations, otherwise I will have 2 jobs scraping my pod.

I found a workaround to disable the annotations scraping. I am overriding the pod annotations using:

spec:
  podAnnotations:
    prometheus.io/scrape: "false"

This works, however it doesn't look clean because now annotations are inconsistent between deployment and pod:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    prometheus.io/path: /metrics
    prometheus.io/port: "8888"
    prometheus.io/scrape: "true"
 ...
spec:
  ...
  template:
    metadata:
      annotations:
        prometheus.io/path: /metrics
        prometheus.io/port: "8888"
        prometheus.io/scrape: "false"
  ...

Describe the solution you'd like

A flag in the collector spec should do the trick to eventually enable/disable these annotations.

Describe alternatives you've considered

A more sophisticated solution would be to give the possibility to choose between these annotations or a servicemonitor created by the opentelemetry operator.

Additional context

Please, ask if it is not clear

jaronoff97 commented 5 months ago

@GiorgioBullo we do already have the option to create a servicemonitor for the collector, I think it would be reasonable to also set this optionally as part of that spec. Would you want to work on this? If not, I can ask around our group and see if anyone can take it.

Toaddyan commented 5 months ago

I can work on this issue