justwatchcom / sql_exporter

Flexible SQL Exporter for Prometheus.
MIT License
393 stars 108 forks source link

Clickhouse http connection support #79

Closed Hubbitus closed 1 year ago

Hubbitus commented 1 year ago

Hello.

I am trying to run exporter with clickhouse, my simple config:

---
jobs:
  - name: "global"
    interval: '5s'
    connections:
      - 'clickhouse://admin:******@10.221.0.19:31673'
    queries:
      - name: "custom_cluster_nodes_count"
        help: "Amount of cluster nodes"
        labels:
          - "cluster"
          - "region"
        values:
          - "amount"
        query:  |
                SELECT 'cluster' as cluster, 'region' as region, COUNT(*) as amount FROM system.clusters

But in logs got:

{"caller":"job.go:190","err":"read tcp 10.0.2.100:60136->10.221.0.19:31673: i/o timeout","job":"global","level":"warn","msg":"Failed to connect","ts":"2023-06-19T08:42:30.770058746Z"}
{"caller":"job.go:190","err":"read tcp 10.0.2.100:59498->10.221.0.19:31673: i/o timeout","job":"global","level":"warn","msg":"Failed to connect","ts":"2023-06-19T08:42:32.334186815Z"}

I've tried use DSN like clickhouse+http://admin:******@10.221.0.19:31673 (by https://github.com/xo/dburl doc), but also without success:

{"caller":"job.go:190","err":"sql: unknown driver \"clickhouse+http\" (forgotten import?)","job":"global","level":"warn","msg":"Failed to connect","ts":"2023-06-19T08:43:45.063663853Z"}

But official clickhouse-go driver also has http support.

How I should configure using http instead of tcp in connection?

dewey commented 1 year ago

Hey, I don't really have time to test this, but I just updated the dependencies. After looking at clickhouse-go it seems like the HTTP protocol if a relatively recent addition (Labeled as experimental in the README) so it might just not have been in the version of the driver that the sql_exporter is using. Could you give the current master a try?

Hubbitus commented 1 year ago

@dewey, thank you very much! But it looks is not enough.

Could you please take a look on https://github.com/justwatchcom/sql_exporter/pull/86?

dewey commented 1 year ago

Thanks, I merged it and will release it today. I'm just figuring out some release process to switch to GitHub registry instead of Docker.

Hubbitus commented 1 year ago

@dewey, thank you again.

I'll wait for published docker images to use.

dewey commented 1 year ago

You're welcome, the images are now available. They are pushed to the GitHub registry and you can get them there: https://github.com/justwatchcom/sql_exporter/pkgs/container/sql_exporter

I tested them internally and we are already running it in production.