search

vernemq / docker-vernemq

VerneMQ Docker image - Starts the VerneMQ MQTT broker and listens on 1883 and 8080 (for websockets).
https://vernemq.com
Apache License 2.0
178 stars 230 forks source link

Prometheus doesn't scrape the /metrics #206

Open rolfberkenbosch opened 4 years ago

rolfberkenbosch commented 4 years ago

Hi All,

Prometheus doesn't scrape the config from vernemq, i have the following helm config:

          image:
            repository: vernemq/vernemq
            tag: 1.10.2-1-alpine
          replicaCount: 1
          serviceMonitor:
            create: true
            enabled: true
          service:
            loadBalancerIP: 192.168.1.200
            type: LoadBalancer
            ws:
              enabled: true
          nodeSelector:
            kubernetes.io/arch: amd64
          persistentVolume:
            enabled: true
            size: 10Gi
            storageClass: "nfs-client"
          ingress:
            enabled: true
            host:
              - vernemq.domainname.nl
            annotations: 
              kubernetes.io/ingress.class: traefik
          statefulset:
            podAnnotations:
              prometheus.io/scrape: "true"
              prometheus.io/port: "8888"
          additionalEnv:
            - name: DOCKER_VERNEMQ_ACCEPT_EULA
              value: "yes"
            - name: DOCKER_VERNEMQ_ALLOW_ANONYMOUS
              value: "on"
kubectl get all -n ha
NAME                                        READY   STATUS    RESTARTS   AGE
pod/vernemq-0                               1/1     Running   0          20m
pod/svclb-vernemq-rh2dx                     2/2     Running   0          6m15s
pod/svclb-vernemq-7kqzq                     2/2     Running   0          3m35s
pod/svclb-vernemq-z5ht2                     2/2     Running   0          3m25s
pod/svclb-vernemq-8zk45                     2/2     Running   0          3m10s

NAME                           TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                         AGE
service/vernemq-headless       ClusterIP      None            <none>          4369/TCP,8888/TCP               70m
service/vernemq                LoadBalancer   10.43.52.31     192.168.1.203   1883:30162/TCP,8080:31046/TCP   70m

NAME                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/svclb-vernemq   4         4         4       4            4           <none>          70m

NAME                       READY   AGE
statefulset.apps/vernemq   1/1     70m

It looks like that everything is running. I'm wondering is the headless service doesn't have an ClusterIP. Is that not needed for prometheus for scraping the /metrics ?

ioolkos commented 4 years ago

@rolfberkenbosch Hm, not sure what's the issue here. But Prometheus certainly needs to be able to contact the VerneMQ nodes to be able to scrape configs (as configured in the Prometheus scrape config).

rolfberkenbosch commented 4 years ago

@ioolkos So it is no issue that the service/vernemq-headless has no ClusterIP address ?

ioolkos commented 4 years ago

@rolfberkenbosch I don't know ;)

a7i commented 4 years ago

@rolfberkenbosch Headless services (by design) do not get an IP address because they do not perform any proxying or load balancing. I am testing this in my cluster right now and will report back with my findings.

https://kubernetes.io/docs/concepts/services-networking/service/#headless-services "For headless Services, a cluster IP is not allocated, kube-proxy does not handle these Services, and there is no load balancing or proxying done by the platform for them"