search

grafana / clickhouse-datasource

Grafana Plugin for ClickHouse
Apache License 2.0
127 stars 58 forks source link

clickhouse datasource plugin.unavailable #907

Open xiaoyuan2019 opened 1 month ago

xiaoyuan2019 commented 1 month ago
SpencerTorres commented 1 month ago

Hello, thanks for submitting an issue.

Do you have any more logs from the grafana process? I see you pasted the HTTP request log, but there should be more details about the error on a different line.

It seems like the plugin process may be failing to start. How are you running Grafana? Which version of Grafana are you running?

xiaoyuan2019 commented 1 month ago

@SpencerTorres Grafana is started using docker-compose, with version v10.4.1. Additionally, when editing the ClickHouse data source in Grafana, there are no more logs available, only that one error log.

SpencerTorres commented 1 month ago

Perhaps you could increase the log level on the grafana deployment to debug? Are you able to update to grafana v11?

xiaoyuan2019 commented 1 month ago

@SpencerTorres grafana debug logs： logger=accesscontrol.evaluator t=2024-07-18T07:22:52.176179059Z level=debug msg="Matched scope" userscope=datasources: targetscope=datasources:uid:eds3rfholibk0a logger=datasources t=2024-07-18T07:22:52.176326787Z level=debug msg="Received command to update data source" url= logger=datasource t=2024-07-18T07:22:52.176346042Z level=debug msg="Applying default URL parsing for this data source type" type=grafana-clickhouse-datasource url= logger=datasource t=2024-07-18T07:22:52.17635266Z level=debug msg="Data source URL doesn't specify protocol, so prepending it with http:// in order to make it unambiguous" type=grafana-clickhouse-datasource url= logger=sqlstore.session t=2024-07-18T07:22:52.177069138Z level=debug msg="reusing existing session" transaction=true logger=sqlstore.session t=2024-07-18T07:22:52.177672094Z level=debug msg="reusing existing session" transaction=true logger=secrets.kvstore t=2024-07-18T07:22:52.185152866Z level=debug msg="got secret value" orgId=1 type=datasource namespace=grafana-clickhouse-datasource logger=sqlstore.session t=2024-07-18T07:22:52.185194628Z level=debug msg="reusing existing session" transaction=true logger=sqlstore.session t=2024-07-18T07:22:53.438429622Z level=debug msg="reusing existing session" transaction=true logger=secrets.kvstore t=2024-07-18T07:22:53.439235618Z level=debug msg="secret value updated" orgId=1 type=datasource namespace=grafana-clickhouse-datasource logger=sqlstore.transactions t=2024-07-18T07:22:53.43927048Z level=debug msg="skip committing the transaction because it belongs to a session created in the outer scope" logger=sqlstore.transactions t=2024-07-18T07:22:53.439287744Z level=debug msg="skip committing the transaction because it belongs to a session created in the outer scope" logger=secrets.kvstore t=2024-07-18T07:22:53.516275566Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=grafana-clickhouse-datasource logger=oauth_token.sync t=2024-07-18T07:22:53.533036664Z level=debug msg="Singleflight request for OAuth token sync" key=user:2 logger=oauthtoken t=2024-07-18T07:22:53.533081266Z level=debug msg="Expiration check has been cached, no need to refresh" userID=2 logger=accesscontrol.service t=2024-07-18T07:22:53.533119396Z level=debug msg="Using cached permissions" key=rbac-permissions-1-user-2 logger=oauth_token.sync t=2024-07-18T07:22:53.718698151Z level=debug msg="Singleflight request for OAuth token sync" key=user:2 logger=oauthtoken t=2024-07-18T07:22:53.718736605Z level=debug msg="Expiration check has been cached, no need to refresh" userID=2 logger=accesscontrol.service t=2024-07-18T07:22:53.718763223Z level=debug msg="Using cached permissions" key=rbac-permissions-1-user-2 logger=accesscontrol.evaluator t=2024-07-18T07:22:53.718952402Z level=debug msg="Matched scope" userscope=datasources: targetscope=datasources:uid:eds3rfholibk0a logger=secrets.kvstore t=2024-07-18T07:22:53.719686488Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=grafana-clickhouse-datasource logger=oauth_token.sync t=2024-07-18T07:22:53.75323206Z level=debug msg="Singleflight request for OAuth token sync" key=user:2 logger=oauthtoken t=2024-07-18T07:22:53.75329547Z level=debug msg="Expiration check has been cached, no need to refresh" userID=2 logger=accesscontrol.service t=2024-07-18T07:22:53.753331696Z level=debug msg="Using cached permissions" key=rbac-permissions-1-user-2 logger=datasources t=2024-07-18T07:22:53.753509997Z level=debug msg="Querying for data source via SQL store" uid=eds3rfholibk0a orgId=1 logger=secrets.kvstore t=2024-07-18T07:22:53.754222911Z level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=grafana-clickhouse-datasource logger=context userId=2 orgId=1 uname=wpsadmin t=2024-07-18T07:22:53.754448183Z level=error msg="Internal server error" error="[plugin.unavailable] plugin unavailable" remote_addr=10.226.11.24 traceID= logger=context userId=2 orgId=1 uname=wpsadmin t=2024-07-18T07:22:53.754502787Z level=error msg="Request Completed" method=GET path=/api/datasources/uid/eds3rfholibk0a/health status=500 remote_addr=10.226.11.24 time_ms=2 duration=2.076036ms size=95 referer=https://10.226.48.76:8081/grafana/connections/datasources/edit/eds3rfholibk0a handler=/api/datasources/uid/:uid/health status_source=server logger=secrets t=2024-07-18T07:22:55.05038341Z level=debug msg="Removing expired data keys from cache..." logger=secrets t=2024-07-18T07:22:55.050429706Z level=debug msg="Removing expired data keys from cache finished successfully"

xiaoyuan2019 commented 1 month ago

@SpencerTorres grafana upgrade to v11.1.0，clickhouse datasource is working，but why？ image

SpencerTorres commented 1 month ago

I'm unsure why the plugin would be unavailable on older versions, the logs don't show any details about why this would be happening. I'll reach out to the other maintainers to see if they know. Thanks for the extra info.

bossinc commented 1 week ago

I'm not able to reproduce this issue using v10.4.1 of Grafana. Maybe restarting Grafana when you switched to v11 fixed this issue and not v11 itself.

SpencerTorres commented 1 week ago

It seems like this could've been a one-off issue related to your specific deployment since we're unable to reproduce the error. Perhaps we should close this and re-open if others have a similar issue

aramperes commented 1 week ago

I ran into this when building on non-amd64 platform i.e. macOS arm64. The Go binary is built with a different name and is not executed on startup.

SpencerTorres commented 1 week ago

I ran into this when building on non-amd64 platform i.e. macOS arm64. The Go binary is built with a different name and is not executed on startup.

The binary has to have a specific name. It must match the configured prefix (gpx_clickhouse, as defined in plugin.json) and the suffix must match the host machine's architecture.

The official plugin build should include binaries for common architectures. I just noticed this issue doesn't include architecture or whether or not this is an official build. Why did it work when upgrading to Grafana v11?

@xiaoyuan2019 please provide these details if this issue is still relevant, otherwise we can close.