grafana / clickhouse-datasource

Grafana Plugin for ClickHouse
Apache License 2.0
122 stars 55 forks source link

Doesn't seem to work with cloudflare analytics and grafana cloud #723

Open ShravanSunder opened 5 months ago

ShravanSunder commented 5 months ago

What happened: I'm trying to connect my grafana cloud instance with clickhouse over https. The clickhouse instance is from cloudflare api.cloudflare.com/client/v4/accounts/xxx/analytics_engine/sql where xxx is the account id.

here is a screen shot 2024-02-21 0229 Brave Browser grafana-clickhouse-datasource - Data sources - Connections - Grafana

2024-02-21 0230 Brave Browser New Issue · grafanaclickhouse-datasource

i've entered my authorization token as specified in the docs

What you expected to happen: I expect the connection to work. instead i see plugin health failed 2024-02-21 0230 Brave Browser New Issue · grafanaclickhouse-datasource

How to reproduce it (as minimally and precisely as possible):

Screenshots

Anything else we need to know?:

Environment:

SpencerTorres commented 5 months ago

Try splitting the values in the URL Cloudflare gives you:

The server address would be api.cloudflare.com and then the HTTP URL Path would be the part after the host: client/v4/accounts/....analytics.... I can't recall whether it needs a leading slash or not, but try it without one first

Let me know if this works for you. The HTTP headers feature was specifically added to support this so I appreciate the feedback.

ShravanSunder commented 5 months ago

@SpencerTorres I split up the url. I get a different error. "connection timed out" when i save and test.

It first says "testing it could take a few minutes" then times out. i don't see any logs.

i tried with and without a leading slash

i'm using the default server port. The cloudflare instructions don't give one, but i'm assuming their using the https ports. I've pinged their 443 and 8443 ports and it seems to be valid.

this is my current setup 2024-02-22 0255 CleanShot CleanShot

ShravanSunder commented 5 months ago

oh i opend dev tools, i do get this health check error from grafana browser console log

2024-02-22 0259 CleanShot CleanShot

SpencerTorres commented 5 months ago

Good idea with the ports, I looked at the other plugin's UI and it seems like Cloudflare would be exposing 443 in this case.

In the network log screenshot you showed, the "response" tab would have more information about the error. My next guess is that this is some kind of auth or permission issue, perhaps similar to this one

ShravanSunder commented 5 months ago

@SpencerTorres the below is for 443. Its a ping health check failed

{
  "message": "Plugin health check failed",
  "messageId": "plugin.healthCheck",
  "statusCode": 500,
  "traceID": "e723f7888bcea5d421af5bd6e0ab30cd"
}
• GET https://askluna.grafana.net/api/datasources/uid/<redacted>/health module.ts:31 G
500 (Internal Server Error)
eval
B.r. <computed>
e. ‹computed>. e. ‹computed>
(anonymous)
(anonymous)
Re. add
(anonymous)
Ae. publishUpdate
(anonymous )
Ae. add
(anonymous)
K
callHealthCheck
testDatasource
(anonymous)
withNoBackendCache
(anonymous )
await in (anonymous) (async)
(anonymous )
fe @ module.ts:31
@ module.ts:31
@ (index):1
backend srv.ts: 131
@ ResponseQueue.ts: 32
@ ResponseQueue. ts: 37
@ FetchQueueWorker.ts: 56
@ FetchQueue. ts: 76
@ FetchQueue. ts:55
@ FetchQueue.ts:59
@ backend srv.ts: 136
@ module.ts: 31
@ DataSourceWithBackend.ts: 352
@ DataSourceWithBackend.ts: 369
@ actions.ts: 147
@ backend srv. ts: 485
@ actions.ts: 145

hooks. ts: 47
EditDataSource. tsx: 136
await in fe (async)

if i use 8443 its an invalid port as you said {"message":"invalid port","status":"ERROR"}

ShravanSunder commented 5 months ago

i'm using the free grafana cloud instance fyi https://grafana.com/get/?pg=graf&plcmt=hero-btn-1

SpencerTorres commented 4 months ago

Still investigating this using my own Cloudflare account. This is the error it's returning on the plugin backend:

Raw error:

{"@level":"error","@message":"clickhouse: unsupported column type \"\\n\\t\\t{\\n\\t\\t\\t\\\"name\\\": \\\"timezone()\\\",\\n\\t\\t\\t\\\"type\\\": \\\"String\\\"\\n\\t\\t}\\n\\t],\\n\\n\\t\\\"data\\\":\\n\\t[\\n\\t\\t{\\n\\t\\t\\t\\\"timezone()\\\":\"","@timestamp":"2024-02-28T01:49:55.600261-05:00"}

Manually formatted/corrected error:

{
  "@level": "error",
  "@message": "clickhouse: unsupported column type",
  "@timestamp": "2024-02-28T01:49:55.600261-05:00",
  "<manuallyFormatted>": {
    "raw": "\"\\n\\t\\t{\\n\\t\\t\\t\\\"name\\\": \\\"timezone()\\\",\\n\\t\\t\\t\\\"type\\\": \\\"String\\\"\\n\\t\\t}\\n\\t],\\n\\n\\t\\\"data\\\":\\n\\t[\\n\\t\\t{\\n\\t\\t\\t\\\"timezone()\\\":\"",
    "formatted": {
      "data": [
        {
          "timezone()": {
            "name": "timezone()",
            "type": "String"
          }
        }
      ]
    }
  }
}

Error string formatted:

{
            "name": "timezone()",
            "type": "String"
        }
    ],

    "data":
    [
        {
            "timezone()":
spelexander commented 2 months ago

I'm also encountering this issue, and would love if there's some solution.

The alternative 3rd party Altinity plugin allows you to query data in Grafana. However the latest version has multiple problems for Cloudflare.

SpencerTorres commented 4 weeks ago

I'm not sure why it would be reporting a String as being unsupported. Maybe their server only allows number types? I do know the plugin automatically runs some queries related to the timezone though. Here's the function it uses: https://clickhouse.com/docs/en/sql-reference/functions/date-time-functions#timezone

I'll need to see where it runs this query, but if we can disable it, then it could work. It seems like your queries would be very limited though if it doesn't allow these other data types.