Disconnect causes "plugin unavailable" error.

[joneal]

I installed the Giolith Websocket plug-in and configured my data source pointing to a custom application with a websocket server. I get data and can render it in Grafana. But when my server is stopped (debugging), the Websocket plug-in can never seem to reconnect. In fact, the only way to get the plug-in to work again is to completely uninstall, re-install, and reconfigure the data source.

I am running Grafana a Docker container. Here are the log messages I receive when the server stops and is restarted.

logger=live.runstream t=2023-03-07T13:40:08.8821326Z level=error msg="Error running stream, re-establishing" path=/ error="error running stream: rpc error: code = Unknown desc = connection Error: dial tcp 192.168.68.135:9991: connect: connection refused" wait=800ms logger=plugin.golioth-websocket-datasource t=2023-03-07T13:40:10.4950601Z level=info msg="RunStream called" Path=/ logger=plugin.golioth-websocket-datasource t=2023-03-07T13:40:10.4951684Z level=info msg="Ws Connect" connectingto=ws://192.168.68.135:9991/ logger=plugin.golioth-websocket-datasource t=2023-03-07T13:40:10.6858652Z level=info msg="Ws Connect" connectedto=ws://192.168.68.135:9991/ logger=plugin.golioth-websocket-datasource t=2023-03-07T13:42:20.1995157Z level=info msg="Read Message routine" detail="closing websocket connection and msgRead channel" logger=live.runstream t=2023-03-07T13:42:20.2046074Z level=error msg="Error running stream, re-establishing" path=/ error="error running stream: rpc error: code = Unavailable desc = error reading from server: EOF" wait=1.6s logger=live.runstream t=2023-03-07T13:42:20.2379297Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=0s logger=live.runstream t=2023-03-07T13:42:20.2490461Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=0s logger=live.runstream t=2023-03-07T13:42:20.460919Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=200ms logger=live.runstream t=2023-03-07T13:42:20.8723485Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=400ms logger=live.runstream t=2023-03-07T13:42:21.6856954Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=800ms logger=live.runstream t=2023-03-07T13:42:23.2978163Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=1.6s logger=live.runstream t=2023-03-07T13:42:26.5325128Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=3.2s logger=context userId=1 orgId=1 uname=admin t=2023-03-07T13:42:29.9604303Z level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr=172.17.0.1 time_ms=33 duration=33.4763ms size=0 referer= handler=/api/live/ws logger=live.features t=2023-03-07T13:42:30.2761713Z level=error msg="Plugin OnSubscribe call error" error="plugin unavailable" path=/ logger=live t=2023-03-07T13:42:30.2762282Z level=error msg="Error calling channel handler subscribe" user=1 client=ad3122c2-bec1-40c1-b27d-0dd23324dfe5 channel=1/ds/-USzCRaVz// error="plugin unavailable" logger=live.runstream t=2023-03-07T13:42:31.5679795Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=5s logger=live.features t=2023-03-07T13:42:32.1847032Z level=error msg="Plugin OnSubscribe call error" error="plugin unavailable" path=/ logger=live t=2023-03-07T13:42:32.1847885Z level=error msg="Error calling channel handler subscribe" user=1 client=ad3122c2-bec1-40c1-b27d-0dd23324dfe5 channel=1/ds/-USzCRaVz// error="plugin unavailable" logger=live.features t=2023-03-07T13:42:34.1933098Z level=error msg="Plugin OnSubscribe call error" error="plugin unavailable" path=/ logger=live t=2023-03-07T13:42:34.1933928Z level=error msg="Error calling channel handler subscribe" user=1 client=ad3122c2-bec1-40c1-b27d-0dd23324dfe5 channel=1/ds/-USzCRaVz// error="plugin unavailable" logger=live.runstream t=2023-03-07T13:42:36.5496082Z level=error msg="Error running stream, re-establishing" path=/ error="plugin unavailable" wait=5s logger=live.features t=2023-03-07T13:42:37.1268603Z level=error msg="Plugin OnSubscribe call error" error="plugin unavailable" path=/ logger=live t=2023-03-07T13:42:37.1269211Z level=error msg="Error calling channel handler subscribe" user=1 client=ad3122c2-bec1-40c1-b27d-0dd23324dfe5 channel=1/ds/-USzCRaVz// error="plugin unavailable" logger=live.features t=2023-03-07T13:42:41.8701649Z level=error msg="Plugin OnSubscribe call error" error="plugin unavailable" path=/ logger=live t=2023-03-07T13:42:41.870216Z level=error msg="Error calling channel handler subscribe" user=1 client=ad3122c2-bec1-40c1-b27d-0dd23324dfe5 channel=1/ds/-USzCRaVz// error="plugin unavailable" logger=live.features t=2023-03-07T13:42:47.8868736Z level=error msg="Plugin OnSubscribe call error" error="plugin unavailable" path=/ logger=live t=2023-03-07T13:42:47.8869443Z level=error msg="Error calling channel handler subscribe" user=1 client=ad3122c2-bec1-40c1-b27d-0dd23324dfe5 channel=1/ds/-USzCRaVz// error="plugin unavailable" logger=live.features t=2023-03-07T13:42:56.1317842Z level=error msg="Plugin OnSubscribe call error" error="plugin unavailable" path=/ logger=live t=2023-03-07T13:42:56.1318718Z level=error msg="Error calling channel handler subscribe" user=1 client=ad3122c2-bec1-40c1-b27d-0dd23324dfe5 channel=1/ds/-USzCRaVz// error="plugin unavailable" logger=live.features t=2023-03-07T13:43:11.3021335Z level=error msg="Plugin OnSubscribe call error" error="plugin unavailable" path=/ logger=live t=2023-03-07T13:43:11.302213Z level=error msg="Error calling channel handler subscribe" user=1 client=ad3122c2-bec1-40c1-b27d-0dd23324dfe5 channel=1/ds/-USzCRaVz// error="plugin unavailable"

I suspect I am doing something wrong. Any suggestions?

[chireia]

Hey @joneal, the only time I saw something like this, it was related to a not supported Grafana version. Which version are you running it on?

I'd ask you to maybe try into another (older) version as well.

[abezkorovayny]

Hey @chireia, I have a similar issue, but I cannot get the plugin to work.

The Grafana version is v9.5.2 (cfcea75916) - Grafana OSS. Besides that, I have the same issue with the Grafana Enterprise, grafana-enterprise-9.3.6.windows-amd64.msi, too.

The log:

logger=context userId=1 orgId=1 uname=admin t=2023-05-17T16:58:37.0946969+02:00 level=error msg="Request Completed" method=POST path=/api/ds/query status=500 remote_addr=[::1] time_ms=31 duration=31.3563ms size=98 referer="http://localhost:3000/explore?left=%7B%22datasource%22%3A%223rIi_s8Vk%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22datasource%22%3A%7B%22type%22%3A%22golioth-websocket-datasource%22%2C%22uid%22%3A%223rIi_s8Vk%22%7D%2C%22path%22%3A%22%2Fws%22%2C%22fields%22%3A%5B%7B%22jsonPath%22%3A%22%22%2C%22language%22%3A%22jsonpath%22%2C%22name%22%3A%22%22%7D%5D%2C%22withStreaming%22%3Atrue%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-1h%22%2C%22to%22%3A%22now%22%7D%7D&orgId=1" handler=/api/ds/query logger=licensing t=2023-05-17T16:59:02.6638598+02:00 level=info msg="Validated license token" appURL=http://localhost:3000/ source=disk status=NotFound logger=licensing.renewal t=2023-05-17T16:59:02.6638598+02:00 level=warn msg="failed to load or validate token" err="license token file not found: C:\Program Files\GrafanaLabs\grafana\data\license.jwt"

Can you check the issue and suggest me a fix? Thank you.

[Waxolunist]

Same with Grafana 10.1.1

[SeanDevanny]

I'm running Grafana Version 10.1.2 (commit: 2b2d40ba32, branch: HEAD) and experiencing the same issue. When disconnecting from server, the plugin dies and needs to be reinstalled