Upgrading to grafana 9.0 causes an error : cannot fetch graph with curl : Operation timed out

[Aleksey-Maksimov]

Hello.

We just updated the grafana 8.5.1 package on our monitoring server to the new version 9.0.1. Plugin in Icinga (1.4.2) stopped showing charts

Performance Graph Debug Access mode indirectproxy Authentication type anon Protocol http Grafana Host mon01.sub.holding.com:3000 Dashboard Store db Dashboard Name icinga2-hostalive Panel ID 2 Organization ID 1 Theme light Timerange now-24h Timerangeto now Height 280 Width 640 Custom Variables
Graph URL /icingaweb2/grafana/img?host=sw029.sub.holding.com&panelid=2&timerange=now-24h&timerangeto=now&cachetime=296 Disable graph custom variable grafana_graph_disable Graph config custom variable grafana_graph_config Shadows No

If I click on the link to the graph, it works and opens

http://mon01.sub.holding.com:3000/d/000000009/icinga2-hostalive?var-hostname=sw029_sub_holding_com&var-service=hostalive4&var-command=hostalive4&from=now-24h&to=now&orgId=1&viewPanel=2

Are there any peculiarities in using the plugin with grafana 9.0?

[Aleksey-Maksimov]

In the grafana log we see this:

logger=rendering renderer=plugin t=2022-06-23T13:51:44.741494085+03:00 level=info msg=Rendering path="d-solo/000000009/icinga2-hostalive?var-hostname=sw029_sub_holding_com&var-service=hostalive4&var-command=hostalive4&panelId=2&orgId=&width=640&height=280&theme=light&from=now-24h&to=now"
logger=plugin.grafana-image-renderer t=2022-06-23T13:51:45.268262127+03:00 level=error msg="Browser uncaught exception" error="Error: SyntaxError: Unexpected token '='"
logger=rendering t=2022-06-23T13:51:54.736854572+03:00 level=error msg="Failed to delete render key" error="context canceled"
logger=context traceID=00000000000000000000000000000000 userId=0 orgId=1 uname= t=2022-06-23T13:51:54.736914472+03:00 level=error msg="Rendering failed." error="rpc error: code = Canceled desc = context canceled"
logger=context traceID=00000000000000000000000000000000 userId=0 orgId=1 uname= t=2022-06-23T13:51:54.737009272+03:00 level=error msg="Request Completed" method=GET path=/render/d-solo/000000009/icinga2-hostalive status=500 remote_addr=10.10.1.20 time_ms=9996 duration=9.996417083s size=1344 referer= traceID=00000000000000000000000000000000
logger=plugin.grafana-image-renderer t=2022-06-23T13:52:46.053296587+03:00 level=error msg="Render request failed" url="http://localhost:3000/d-solo/000000009/icinga2-hostalive?var-hostname=sw029_sub_holding_com&var-service=hostalive4&var-command=hostalive4&panelId=2&orgId=&width=640&height=280&theme=light&from=now-24h&to=now&render=1" error="TimeoutError: waiting for function failed: timeout 60000ms exceeded"

[Mikesch-mp]

i will test it this week. need to upgrade my webservers from centos 8 first

[Mikesch-mp]

on my "old" setup, now CentOS8 Stream it works without problems with Grafana 9.0.1. But iam still on icingaweb 2.9.5, have to test if it comes from that.

Did you upgrade also your Grafana plugins with ´grafana-cli plugins upgrade-all` ?

[Aleksey-Maksimov]

You are right Carsten

I haven't updated the rendering plugin. Thank you.