search

artemiscloud / activemq-artemis-operator

Apache License 2.0
63 stars 62 forks source link

Using ServiceMonitor with Prometheus Operator is not possible because of duplicate services with same labels #978

Closed johgoe closed 6 days ago

johgoe commented 2 months ago

My cluster contains configuration like this

apiVersion: broker.amq.io/v1beta1
kind: ActiveMQArtemis
metadata:
  name: active-mq-artemis-test
spec:
  deploymentPlan:
    enableMetricsPlugin: true
    size: 2
  acceptors:
    - name: http-core
      port: 61618
      protocols: CORE

which causes an additional service per broker node.

Because these services and hdls and the ping service contains the application label the following servicemonitor will cause duplicate scraper

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: artemis-with-metrics-monitor
spec:
  selector:
    matchLabels:
      application: active-mq-artemis-test
  endpoints:
  - targetPort: 8161

My cluster with 2 nodes are watched by 6 scrapers because of these. These are the labels shown in prometheus:

I guess currently there is no better way, but maybe an additional label like app.kubernetes.io/component: headless can be added and used within the serviceMonitor

[!TIP] Vote this issue reacting with :+1: or :-1:

johgoe commented 2 months ago

@brusdev fyi, I saw you are currently writing an example how to use it.

johgoe commented 2 months ago

A workaround which works:

apiVersion: broker.amq.io/v1beta1
kind: ActiveMQArtemis
metadata:
  name: active-mq-artemis-test
  deploymentPlan:
    enableMetricsPlugin: true
    size: 2
  resourceTemplates:
    - selector:
        kind: "Service"
        name: "active-mq-artemis-test-hdls-svc"
      labels:
        app.kubernetes.io/component: headless

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: artemis-with-metrics-monitor
spec:
  selector:
    matchLabels:
      application: active-mq-artemis-test
      app.kubernetes.io/component: headless
  endpoints:
  - targetPort: 8161
brusdev commented 2 months ago

@johgoe thanks for the heads-up and sharing your workaround. The operator could add a specific label to the headless by default.

brusdev commented 1 month ago

@johgoe I'm not able to repduce this issue with my example because that example use the port name wconsj that is only exposed by the console service while you are using the port number 8161.

brusdev commented 1 month ago

I have just updated my example to use the port console-jolokia that exists even if the console is not exposed, see #977

johgoe commented 1 month ago

I didn't expose the console that's why wconsj didn't exists. I will try port console-jokokia instead of 8161