metrics via enabling embedded exporter VS. metrics offered by the "clickhouse-operator-metrics" service

[madhur-df]

The guide here: https://kb.altinity.com/altinity-kb-setup-and-maintenance/altinity-kb-monitoring suggests many ways of exposing metrics.

The first one is enabling the embedded exporter at port 9363 i.e.

    <prometheus>
        <endpoint>/metrics</endpoint>
        <port>9363</port>
        <metrics>true</metrics>
        <events>true</events>
        <asynchronous_metrics>true</asynchronous_metrics>
        <errors>true</errors>
    </prometheus>

which can be done via the following:

  configuration:
    settings:
      prometheus/endpoint: /metrics
      prometheus/port: 9363
      prometheus/metrics: true
      prometheus/events: true
      prometheus/asynchronous_metrics: true
      prometheus/errors: true
      prometheus/status_info: true

The second one is using clickhouse-operator-metrics service present in the kube-system namespace, which has metrics at two ports -- 8888 for clickhouse-metrics and 9999 for operator-metrics.

I wanted to ask if metrics at 9363 exposed via the first method is same as metrics at 8888 exposed via the service.

Please let me know what is the difference...

[Slach]

Most of the metrics shall be the same, labels will difference a bit no hostname label and different namespace label based on your prometheus scraping config

You could install both the methods and try to compare curl http://clickhouse-operator-metrics.kube-system:8888/metrics and curl chi-{chi-name}-{cluster-name}-0-0.your-namespace:9363/metrics