koelle25 commented 1 month ago

What happened: I just updated the plugin from 3.11.0 to 3.11.1, now the rendering doesn't work anymore. Browser shows this image instead: grafik

In the grafana.log I have this:

logger=grafana.update.checker t=2024-07-16T15:47:04.283135203+02:00 level=info msg="Update check succeeded" duration=234.799768ms
logger=plugin.grafana-image-renderer t=2024-07-16T15:47:05.887656459+02:00 level=error msg="Failed to get browser version" err=map[]
logger=plugins.update.checker t=2024-07-16T15:47:06.028639677+02:00 level=info msg="Update check succeeded" duration=1.97984127s
logger=rendering renderer=plugin t=2024-07-16T15:47:06.083086254+02:00 level=info msg=Rendering path="d-solo/qDwRNfbik/icinga2-with-influxdb?var-hostname=jh.vms.idial.fh&var-service=hostalive4&var-command=hostalive4&panelId=1&orgId=&width=640&height=320&theme=light&from=now-6h&to=now" userID=4
logger=grafana-apiserver t=2024-07-16T15:47:06.15364003+02:00 level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager"
logger=grafana-apiserver t=2024-07-16T15:47:06.154718455+02:00 level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager"
logger=plugin.grafana-image-renderer t=2024-07-16T15:47:06.309327865+02:00 level=error msg="Render request failed" url="http://localhost:3000/d-solo/qDwRNfbik/icinga2-with-influxdb?var-hostname=jh.vms.idial.fh&var-service=hostalive4&var-command=hostalive4&panelId=1&orgId=&width=640&height=320&theme=light&from=now-6h&to=now&render=1" error="Error: Failed to launch the browser process!\nchrome_crashpad_handler: --database is required\nTry 'chrome_crashpad_handler --help' for more information.\n[3002066:3002066:0716/154706.161379:ERROR:socket.cc(120)] recvmsg: Connection reset by peer (104)\n\n\nTROUBLESHOOTING: https://pptr.dev/troubleshooting\n"
logger=context userId=4 orgId=1 uname=sa-1-autogen-1-icinga2 t=2024-07-16T15:47:06.312963968+02:00 level=error msg="Rendering failed." error="rendering failed: Error: Failed to launch the browser process!\nchrome_crashpad_handler: --database is required\nTry 'chrome_crashpad_handler --help' for more information.\n[3002066:3002066:0716/154706.161379:ERROR:socket.cc(120)] recvmsg: Connection reset by peer (104)\n\n\nTROUBLESHOOTING: https://pptr.dev/troubleshooting\n"
logger=context userId=4 orgId=1 uname=sa-1-autogen-1-icinga2 t=2024-07-16T15:47:06.314198587+02:00 level=error msg="Request error" error="Context.HTML - Error rendering template: error. You may need to build frontend assets \n template: error:16:42: executing \"error\" at <.Assets.Dark>: can't evaluate field Assets in type struct { Title string; AppTitle string; AppSubUrl string; ThemeType string; ErrorMsg error }" stack="github.com/grafana/grafana/pkg/web/context.go:112 (0x74e8064)\ngithub.com/grafana/grafana/pkg/services/contexthandler/model/model.go:54 (0x74f3328)\ngithub.com/grafana/grafana/pkg/api/render.go:106 (0xa44612a)\ngithub.com/grafana/grafana/pkg/api/response/web_hack.go:40 (0x74f94af)\nnet/http/server.go:2166 (0x74e90f8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:131 (0x74e90b5)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/services/contexthandler/contexthandler.go:141 (0xa4505f4)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/render.go:44 (0x74e9c0b)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/csrf/csrf.go:66 (0x98f45af)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/recovery.go:180 (0xa428ffd)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/loggermw/logger.go:72 (0x98f553e)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/request_metrics.go:75 (0x82b19ed)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/request_tracing.go:88 (0xa429d75)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/requestmeta/request_metadata.go:66 (0xa44f6e1)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/context.go:52 (0x74e79fa)\ngithub.com/grafana/grafana/pkg/web/router.go:155 (0x74eb3ed)\ngithub.com/grafana/grafana/pkg/web/router.go:221 (0x74ebf14)\ngithub.com/grafana/grafana/pkg/web/macaron.go:163 (0x74e95dc)\nnet/http/server.go:3137 (0x624272d)\nnet/http/server.go:2039 (0x623da07)\nruntime/asm_amd64.s:1695 (0x5ee74c0)\n"
logger=context userId=4 orgId=1 uname=sa-1-autogen-1-icinga2 t=2024-07-16T15:47:06.314452972+02:00 level=error msg="Request Completed" method=GET path=/render/d-solo/qDwRNfbik/icinga2-with-influxdb status=500 remote_addr=127.0.0.1 time_ms=244 duration=244.168582ms size=351 referer= handler=/render/* status_source=server

After a downgrade to 3.11.0 everything is working again.

What you expected to happen: Rendering still works as before.

How to reproduce it (as minimally and precisely as possible):

kevin@icinga:~$ sudo systemctl stop grafana-server.service
kevin@icinga:~$ sudo -u grafana -H bash
grafana@icinga:/home/kevin$ cd /storage/grafana/plugins/
grafana@icinga:/storage/grafana/plugins$ rm -rf grafana-image-renderer/
grafana@icinga:/storage/grafana/plugins$ wget https://github.com/grafana/grafana-image-renderer/releases/download/v3.11.1/plugin-linux-x64-glibc.zip
grafana@icinga:/storage/grafana/plugins$ unzip plugin-linux-x64-glibc.zip
grafana@icinga:/storage/grafana/plugins$ rm plugin-linux-x64-glibc.zip
grafana@icinga:/storage/grafana/plugins$ mv plugin-linux-x64-glibc/ grafana-image-renderer/
grafana@icinga:/storage/grafana/plugins$ exit
kevin@icinga:~$ sudo systemctl start grafana-server.service

Anything else we need to know?:

Environment:

Grafana Image Renderer version: 3.11.0 -> 3.11.1
Grafana version: 11.1.0
Installed plugin or remote renderer service: Installed (custom plugin directory defined in /etc/grafana/grafana.ini: plugins = /storage/grafana/plugins)
OS Grafana Image Renderer is installed on: Ubuntu 20.04
User OS & Browser: Windows 11 23H2, Firefox 115.13.0esr
Others:

Trolls commented 1 month ago

the same for me, i just updated the plugin from 3.11.0 to 3.11.1, i got the same issue. OS : Centos 8

AgnesToulet commented 1 month ago

Hello! I wasn't able to reproduce it on Ubuntu 20.04. Could you enable more logging to help me understand the issue?

You can update the following settings to have more logs on start-up:

[plugin.grafana-image-renderer]
rendering_verbose_logging = true
rendering_dumpio = true

[log]
level=debug

koelle25 commented 1 month ago

This is with debug logging enabled:

Directly after service start:

logger=plugin.grafana-image-renderer t=2024-07-17T13:39:19.235455027+02:00 level=debug msg="Setting chromeBin to /home/idial/icinga2/grafana-plugins/grafana-image-renderer_3.11.1/chrome/linux-128.0.6597.0/chrome-linux64/chrome"
logger=plugin.grafana-image-renderer t=2024-07-17T13:39:19.238606939+02:00 level=debug msg="Browser initialized" config="map[args:[--no-sandbox --disable-gpu] chromeBin:/home/idial/icinga2/grafana-plugins/grafana-image-renderer_3.11.1/chrome/linux-128.0.6597.0/chrome-linux64/chrome clustering:map[maxConcurrency:5 mode:browser monitor:false timeout:30] deviceScaleFactor:1 dumpio:true emulateNetworkConditions:false headed:false height:500 ignoresHttpsErrors:false maxDeviceScaleFactor:4 maxHeight:3000 maxWidth:3000 mode:default pageZoomLevel:1 timingMetrics:false verboseLogging:false width:1000]"
logger=plugin.grafana-image-renderer t=2024-07-17T13:39:19.280660299+02:00 level=debug msg="chrome_crashpad_handler: --database is required"
logger=plugin.grafana-image-renderer t=2024-07-17T13:39:19.282079551+02:00 level=debug msg="Try 'chrome_crashpad_handler --help' for more information."
logger=plugin.grafana-image-renderer t=2024-07-17T13:39:19.282791358+02:00 level=debug msg="[3526327:3526327:0717/133919.282542:ERROR:socket.cc(120)] recvmsg: Connection reset by peer (104)"
logger=plugin.grafana-image-renderer t=2024-07-17T13:39:19.432299405+02:00 level=error msg="Failed to get browser version" err=map[]
logger=plugin.grafana-image-renderer t=2024-07-17T13:39:19.436334729+02:00 level=debug msg="using plugin" version=2

After a render request:

logger=rendering renderer=plugin t=2024-07-17T13:44:32.853553465+02:00 level=info msg=Rendering path="d-solo/Pcey9Axiz/icinga2-default?var-hostname=sems.vms.idial.fh&var-service=apt&var-command=apt&panelId=1&orgId=1&width=640&height=320&theme=light&from=now-6h&to=now" userID=4
logger=rendering renderer=plugin t=2024-07-17T13:44:32.860598068+02:00 level=debug msg="Calling renderer plugin" req="url:\"http://localhost:3000/d-solo/Pcey9Axiz/icinga2-default?var-hostname=sems.vms.idial.fh&var-service=apt&var-command=apt&panelId=1&orgId=1&width=640&height=320&theme=light&from=now-6h&to=now&render=1\" width:640 height:320 deviceScaleFactor:1 filePath:\"/storage/grafana/png/Af4F0hda7mDlKnGuZevy.png\" renderKey:\"OCcOepKfJfkdEmb99AHDFsgecjjydtkz\" domain:\"localhost\" timeout:60 authToken:\"-\" encoding:\"png\""
logger=plugin.grafana-image-renderer t=2024-07-17T13:44:32.866667187+02:00 level=debug msg="Render request received" url="http://localhost:3000/d-solo/Pcey9Axiz/icinga2-default?var-hostname=sems.vms.idial.fh&var-service=apt&var-command=apt&panelId=1&orgId=1&width=640&height=320&theme=light&from=now-6h&to=now&render=1"
logger=plugin.grafana-image-renderer t=2024-07-17T13:44:32.888890849+02:00 level=debug msg="chrome_crashpad_handler: --database is required"
logger=plugin.grafana-image-renderer t=2024-07-17T13:44:32.88937929+02:00 level=debug msg="Try 'chrome_crashpad_handler --help' for more information."
logger=plugin.grafana-image-renderer t=2024-07-17T13:44:32.889971804+02:00 level=debug msg="[3528378:3528378:0717/134432.889675:ERROR:socket.cc(120)] recvmsg: Connection reset by peer (104)"
logger=plugin.grafana-image-renderer t=2024-07-17T13:44:33.042594291+02:00 level=error msg="Render request failed" url="http://localhost:3000/d-solo/Pcey9Axiz/icinga2-default?var-hostname=sems.vms.idial.fh&var-service=apt&var-command=apt&panelId=1&orgId=1&width=640&height=320&theme=light&from=now-6h&to=now&render=1" error="Error: Failed to launch the browser process!\nchrome_crashpad_handler: --database is required\nTry 'chrome_crashpad_handler --help' for more information.\n[3528378:3528378:0717/134432.889675:ERROR:socket.cc(120)] recvmsg: Connection reset by peer (104)\n\n\nTROUBLESHOOTING: https://pptr.dev/troubleshooting\n"
logger=context userId=4 orgId=1 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:44:33.04909176+02:00 level=error msg="Rendering failed." error="rendering failed: Error: Failed to launch the browser process!\nchrome_crashpad_handler: --database is required\nTry 'chrome_crashpad_handler --help' for more information.\n[3528378:3528378:0717/134432.889675:ERROR:socket.cc(120)] recvmsg: Connection reset by peer (104)\n\n\nTROUBLESHOOTING: https://pptr.dev/troubleshooting\n"
logger=context userId=4 orgId=1 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:44:33.050074942+02:00 level=error msg="Request error" error="Context.HTML - Error rendering template: error. You may need to build frontend assets \n template: error:16:42: executing \"error\" at <.Assets.Dark>: can't evaluate field Assets in type struct { Title string; AppTitle string; AppSubUrl string; ThemeType string; ErrorMsg error }" stack="github.com/grafana/grafana/pkg/web/context.go:112 (0x74e8064)\ngithub.com/grafana/grafana/pkg/services/contexthandler/model/model.go:54 (0x74f3328)\ngithub.com/grafana/grafana/pkg/api/render.go:106 (0xa44612a)\ngithub.com/grafana/grafana/pkg/api/response/web_hack.go:40 (0x74f94af)\nnet/http/server.go:2166 (0x74e90f8)\ngithub.com/grafana/grafana/pkg/web/macaron.go:131 (0x74e90b5)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/services/contexthandler/contexthandler.go:141 (0xa4505f4)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/render.go:44 (0x74e9c0b)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/macaron.go:137 (0x74e9194)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/csrf/csrf.go:66 (0x98f45af)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/recovery.go:180 (0xa428ffd)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/loggermw/logger.go:72 (0x98f553e)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/request_metrics.go:75 (0x82b19ed)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/request_tracing.go:88 (0xa429d75)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/middleware/requestmeta/request_metadata.go:66 (0xa44f6e1)\nnet/http/server.go:2166 (0x623f028)\ngithub.com/grafana/grafana/pkg/web/context.go:52 (0x74e79fa)\ngithub.com/grafana/grafana/pkg/web/router.go:155 (0x74eb3ed)\ngithub.com/grafana/grafana/pkg/web/router.go:221 (0x74ebf14)\ngithub.com/grafana/grafana/pkg/web/macaron.go:163 (0x74e95dc)\nnet/http/server.go:3137 (0x624272d)\nnet/http/server.go:2039 (0x623da07)\nruntime/asm_amd64.s:1695 (0x5ee74c0)\n"
logger=context userId=4 orgId=1 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:44:33.050285367+02:00 level=error msg="Request Completed" method=GET path=/render/d-solo/Pcey9Axiz/icinga2-default status=500 remote_addr=127.0.0.1 time_ms=211 duration=211.36366ms size=351 referer= handler=/render/* status_source=server

Note that /storage/grafana/plugins/ is actually a symlink to /home/idial/icinga2/grafana-plugins/. But this shouldn't be a problem, as is worked with any version before.

koelle25 commented 1 month ago

Looking through the log myself, could it be this?

template: error:16:42: executing \"error\" at <.Assets.Dark>: can't evaluate field Assets in type struct { Title string; AppTitle string; AppSubUrl string; ThemeType string; ErrorMsg error }

koelle25 commented 1 month ago

These are the debug logs with 3.11.0 plugin:

Directly after start:

``` Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.664385575+02:00 level=debug msg="Setting chromeBin to /home/idial/icinga2/grafana-plugins/grafana-image-renderer_3.11.0/chrome/linux-128.0.6534.0/chrome-linux64/chrome" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.667509307+02:00 level=debug msg="Browser initialized" config="map[args:[--no-sandbox --disable-gpu] chromeBin:/home/idial/icinga2/grafana-plugins/grafana-image-renderer_3.11.0/chrome/linux-128.0.6534.0/chrome-linux64/chrome clustering:map[maxConcurrency:5 mode:browser monitor:false timeout:30] deviceScaleFactor:1 dumpio:true emulateNetworkConditions:false headed:false height:500 ignoresHttpsErrors:false maxDeviceScaleFactor:4 maxHeight:3000 maxWidth:3000 mode:default pageZoomLevel:1 timingMetrics:false verboseLogging:false width:1000]" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.743369945+02:00 level=debug msg="[0717/134900.741956:ERROR:address_tracker_linux.cc(216)] Could not create NETLINK socket: Address family not supported by protocol (97)" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.744342457+02:00 level=debug msg="[0717/134900.744264:ERROR:udev_watcher.cc(51)] Failed to initialize a udev monitor." Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.749842063+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.750304124+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.753721633+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.754287406+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.754690575+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.755194227+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.755597146+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.756025745+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.758438632+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.759656719+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.765934664+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.766768203+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.767681464+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.768161805+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.768541034+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.786958828+02:00 level=debug msg="[0717/134900.786694:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable." Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.796848526+02:00 level=debug msg= Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.797127502+02:00 level=debug msg="DevTools listening on ws://127.0.0.1:36863/devtools/browser/3c3701ae-3597-4f2f-b915-60d15d6859d5" Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.82957772+02:00 level=debug msg="[0717/134900.807429:WARNING:sandbox_linux.cc(430)] InitializeSandbox() called with multiple threads in process gpu-process." Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.904448955+02:00 level=debug msg="Using browser version" browserVersion=HeadlessChrome/128.0.6534.0 Jul 17 13:49:00 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:49:00.90729201+02:00 level=debug msg="using plugin" version=2 ```

After a render request:

``` Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=rendering renderer=plugin t=2024-07-17T13:52:16.798260841+02:00 level=info msg=Rendering path="d-solo/Pcey9Axiz/icinga2-default?var-hostname=sems.vms.idial.fh&var-service=apt&var-command=apt&panelId=1&orgId=1&width=640&height=320&theme=light&from=now-6h&to=now" userID=4 Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=rendering renderer=plugin t=2024-07-17T13:52:16.802611961+02:00 level=debug msg="Calling renderer plugin" req="url:\"http://localhost:3000/d-solo/Pcey9Axiz/icinga2-default?var-hostname=sems.vms.idial.fh&var-service=apt&var-command=apt&panelId=1&orgId=1&width=640&height=320&theme=light&from=now-6h&to=now&render=1\" width:640 height:320 deviceScaleFactor:1 filePath:\"/storage/grafana/png/aRyc0WSzxyLBPm9nfAEO.png\" renderKey:\"0aBJq1Tj3uWfvEtQPhyZf4p6RBYxR5nJ\" domain:\"localhost\" timeout:60 authToken:\"-\" encoding:\"png\"" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.80473198+02:00 level=debug msg="Render request received" url="http://localhost:3000/d-solo/Pcey9Axiz/icinga2-default?var-hostname=sems.vms.idial.fh&var-service=apt&var-command=apt&panelId=1&orgId=1&width=640&height=320&theme=light&from=now-6h&to=now&render=1" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.872296605+02:00 level=debug msg="[0717/135216.871915:ERROR:address_tracker_linux.cc(216)] Could not create NETLINK socket: Address family not supported by protocol (97)" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.875208483+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.876567604+02:00 level=debug msg="[0717/135216.876458:ERROR:udev_watcher.cc(51)] Failed to initialize a udev monitor." Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.880044974+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.880968396+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.881390305+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.881751754+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.88504944+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.885457029+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.885823838+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.888215072+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.888595581+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.893236208+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.895830718+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.896279368+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.896634026+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.897051336+02:00 level=debug msg="Fontconfig error: No writable cache directories" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.921299314+02:00 level=debug msg="[0717/135216.920988:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable." Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.928740746+02:00 level=debug msg= Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.929032272+02:00 level=debug msg="DevTools listening on ws://127.0.0.1:33973/devtools/browser/be128a6b-6643-4e80-a3c0-016df7764027" Jul 17 13:52:16 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:16.940063447+02:00 level=debug msg="[0717/135216.939883:WARNING:sandbox_linux.cc(430)] InitializeSandbox() called with multiple threads in process gpu-process." Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.039505046+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(licensing:read server.stats:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.041232786+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(licensing:read server.stats:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:17.040872188+02:00 level=debug msg="[0717/135217.036123:ERROR:socket_posix.cc(93)] CreatePlatformSocket() failed: Address family not supported by protocol (97)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.042169837+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:read scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.042779392+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(folders:read folders:create dashboards:read dashboards:create)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.042969166+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:create scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.04313353+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:create scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.043284553+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:datasources:explore scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.043437266+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:datasources:explore scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.04362176+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(alert.rules:read alert.rules.external:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.043868547+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(alert.notifications:read alert.notifications.external:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.044032981+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(alert.instances:read alert.instances.external:read alert.silences:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.044251645+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(alert.instances:read alert.instances.external:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.044411969+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(alert.rules:create alert.rules.external:write)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.044556512+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(alert.rules:create alert.rules.external:write)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.044715226+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(datasources:explore datasources:create all of datasources:read, any of datasources:delete, datasources:write)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.044904471+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(datasources:explore datasources:create all of datasources:read, any of datasources:delete, datasources:write)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.045070644+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(plugins:write plugins:install)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.045773731+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(plugins:write plugins:install)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.045931534+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(org.users:read users:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.046095778+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(org.users:read users:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.046251351+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(teams:create all of teams:read, any of teams:write, teams.permissions:write, teams.permissions:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.046405145+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(teams:create all of teams:read, any of teams:write, teams.permissions:write, teams.permissions:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.046555678+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(serviceaccounts:read serviceaccounts:create)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.046714952+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(serviceaccounts:read serviceaccounts:create)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.047136592+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:apikeys:read scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.047287855+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:apikeys:read scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.047437229+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(all of orgs:read, orgs:write all of orgs.preferences:read, orgs.preferences:write)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.047595062+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(all of orgs:read, orgs:write all of orgs.preferences:read, orgs.preferences:write)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.047747536+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(all of settings:write, settings:read ldap.status:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.0479001+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(all of settings:write, settings:read ldap.status:read)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.048069544+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(settings:read settings:write settings:read settings:write settings:read settings:write settings:read settings:write settings:read settings:write settings:read settings:write)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.048231647+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:settings:read scopes:settings:*" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.05092603+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=0 permissions="action:orgs:read scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.052134037+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=0 permissions="action:orgs:read scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.052298351+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:datasources:read scopes:" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.052451264+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:settings:read scopes:settings:*" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.052607868+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(support.bundles:read support.bundles:create)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.052760032+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(support.bundles:read support.bundles:create)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.053154971+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="any(dashboards:create folders:create)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:17.053332425+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="any(dashboards:create folders:create)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:17.065234489+02:00 level=debug msg="[0717/135217.064895:ERROR:socket_posix.cc(93)] CreatePlatformSocket() failed: Address family not supported by protocol (97)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:17.105272521+02:00 level=debug msg="[0717/135217.105064:ERROR:socket_posix.cc(93)] CreatePlatformSocket() failed: Address family not supported by protocol (97)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:17.110314757+02:00 level=debug msg="[0717/135217.110175:ERROR:socket_posix.cc(93)] CreatePlatformSocket() failed: Address family not supported by protocol (97)" Jul 17 13:52:17 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:17.117968833+02:00 level=debug msg="[0717/135217.117767:ERROR:socket_posix.cc(93)] CreatePlatformSocket() failed: Address family not supported by protocol (97)" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=plugin.grafana-image-renderer t=2024-07-17T13:52:18.161973074+02:00 level=debug msg="[0717/135218.161511:ERROR:socket_posix.cc(93)] CreatePlatformSocket() failed: Address family not supported by protocol (97)" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=context userId=4 orgId=1 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.165999456+02:00 level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr=127.0.0.1 time_ms=3 duration=3.04652ms size=0 referer= handler=/api/live/ws status_source=server Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=live t=2024-07-17T13:52:18.34291289+02:00 level=debug msg="Client connected" user=4 client=6922e0bf-02d7-47e5-9851-d597c6123cfc Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.575312423+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:read scopes:" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.576469079+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:read scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.576626053+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:write scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.577461322+02:00 level=debug msg="Resolved scope" scope=dashboards:uid:Pcey9Axiz resolved_scopes="[dashboards:uid:Pcey9Axiz folders:uid:general]" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.577620975+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:write scopes:dashboards:uid:Pcey9Axiz, folders:uid:general" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=dashboard.permissions t=2024-07-17T13:52:18.577771309+02:00 level=debug msg="Access denied to dashboard" namespaceID=user userId=4 id=4 permissions="action:dashboards:write scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.577931872+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:write scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.578089626+02:00 level=debug msg="Used cache to resolve scope" scope=dashboards:uid:Pcey9Axiz resolved_scopes="[dashboards:uid:Pcey9Axiz folders:uid:general]" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.578237829+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:write scopes:dashboards:uid:Pcey9Axiz, folders:uid:general" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=dashboard.permissions t=2024-07-17T13:52:18.578390523+02:00 level=debug msg="Access denied to dashboard" namespaceID=user userId=4 id=4 permissions="action:dashboards:write scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.578719721+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="all(action:dashboards.permissions:read scopes:dashboards:uid:Pcey9Axiz action:dashboards.permissions:write scopes:dashboards:uid:Pcey9Axiz)" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.578873994+02:00 level=debug msg="Used cache to resolve scope" scope=dashboards:uid:Pcey9Axiz resolved_scopes="[dashboards:uid:Pcey9Axiz folders:uid:general]" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.579048308+02:00 level=debug msg="Used cache to resolve scope" scope=dashboards:uid:Pcey9Axiz resolved_scopes="[dashboards:uid:Pcey9Axiz folders:uid:general]" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.579199211+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="all(action:dashboards.permissions:read scopes:dashboards:uid:Pcey9Axiz, folders:uid:general action:dashboards.permissions:write scopes:dashboards:uid:Pcey9Axiz, folders:uid:general)" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=dashboard.permissions t=2024-07-17T13:52:18.579351956+02:00 level=debug msg="Access denied to dashboard" namespaceID=user userId=4 id=4 permissions="all(action:dashboards.permissions:read scopes:dashboards:uid:Pcey9Axiz action:dashboards.permissions:write scopes:dashboards:uid:Pcey9Axiz)" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.579501319+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:delete scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.579650812+02:00 level=debug msg="Used cache to resolve scope" scope=dashboards:uid:Pcey9Axiz resolved_scopes="[dashboards:uid:Pcey9Axiz folders:uid:general]" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.579815896+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:delete scopes:dashboards:uid:Pcey9Axiz, folders:uid:general" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=dashboard.permissions t=2024-07-17T13:52:18.579969109+02:00 level=debug msg="Access denied to dashboard" namespaceID=user userId=4 id=4 permissions="action:dashboards:delete scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.582838685+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:annotations:create scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.583109541+02:00 level=debug msg="Used cache to resolve scope" scope=dashboards:uid:Pcey9Axiz resolved_scopes="[dashboards:uid:Pcey9Axiz folders:uid:general]" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.583264635+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:annotations:create scopes:dashboards:uid:Pcey9Axiz, folders:uid:general" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.583419449+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:annotations:delete scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.583689145+02:00 level=debug msg="Used cache to resolve scope" scope=dashboards:uid:Pcey9Axiz resolved_scopes="[dashboards:uid:Pcey9Axiz folders:uid:general]" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.583883599+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:annotations:delete scopes:dashboards:uid:Pcey9Axiz, folders:uid:general" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.584153695+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:annotations:write scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.584302008+02:00 level=debug msg="Used cache to resolve scope" scope=dashboards:uid:Pcey9Axiz resolved_scopes="[dashboards:uid:Pcey9Axiz folders:uid:general]" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.584555155+02:00 level=debug msg="Evaluating resolved permissions" namespace=user id=4 orgID=1 permissions="action:annotations:write scopes:dashboards:uid:Pcey9Axiz, folders:uid:general" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.584708438+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:annotations:create scopes:annotations:type:organization" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.584861792+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:annotations:delete scopes:annotations:type:organization" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.585144308+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:annotations:write scopes:annotations:type:organization" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.749071193+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:datasources:query scopes:" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=datasources t=2024-07-17T13:52:18.750906706+02:00 level=debug msg="Querying for data source via SQL store" uid=000000001 orgId=1 Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=query_data t=2024-07-17T13:52:18.751745515+02:00 level=debug msg="Processed metrics query" ref_id=metricFindQuery from=0 to=0 interval=1000 max_data_points=1000 query="{\"adhocFilters\":[],\"alias\":\"\",\"datasource\":{\"type\":\"influxdb\",\"uid\":\"000000001\"},\"datasourceId\":1,\"limit\":\"\",\"maxDataPoints\":1000,\"measurement\":\"\",\"policy\":\"\",\"query\":\"SHOW TAG VALUES WITH KEY = \\\"hostname\\\"\",\"rawQuery\":true,\"rawSql\":\"\",\"refId\":\"metricFindQuery\",\"slimit\":\"\",\"tz\":\"\"}" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=secrets.kvstore t=2024-07-17T13:52:18.752352319+02:00 level=debug msg="got secret value" orgId=1 type=datasource namespace=influxdb Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influxdb endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.752588215+02:00 level=debug msg="Received a query request" numQueries=1 Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influxdb endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.752749098+02:00 level=debug msg="Making a InfluxQL type query" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.752946183+02:00 level=debug msg="Influxdb query" rawquery="SHOW TAG VALUES WITH KEY = \"hostname\"" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.753122717+02:00 level=debug msg="Influxdb request" url="http://localhost:8086/query?db=icinga2&epoch=ms&q=SHOW+TAG+VALUES++WITH+KEY+%3D+%22hostname%22" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.776430424+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:datasources:query scopes:" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=query_data t=2024-07-17T13:52:18.777246732+02:00 level=debug msg="Processed metrics query" ref_id=metricFindQuery from=0 to=0 interval=1000 max_data_points=1000 query="{\"adhocFilters\":[],\"alias\":\"\",\"datasource\":{\"type\":\"influxdb\",\"uid\":\"000000001\"},\"datasourceId\":1,\"limit\":\"\",\"maxDataPoints\":1000,\"measurement\":\"\",\"policy\":\"\",\"query\":\"SHOW TAG VALUES WITH KEY = \\\"service\\\" WHERE hostname =~ /^sems.vms.idial.fh$/\",\"rawQuery\":true,\"rawSql\":\"\",\"refId\":\"metricFindQuery\",\"slimit\":\"\",\"tz\":\"\"}" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=secrets.kvstore t=2024-07-17T13:52:18.777456817+02:00 level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=influxdb Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influxdb endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.777651842+02:00 level=debug msg="Received a query request" numQueries=1 Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influxdb endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.777806725+02:00 level=debug msg="Making a InfluxQL type query" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.777978819+02:00 level=debug msg="Influxdb query" rawquery="SHOW TAG VALUES WITH KEY = \"service\" WHERE hostname =~ /^sems.vms.idial.fh$/" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.778157543+02:00 level=debug msg="Influxdb request" url="http://localhost:8086/query?db=icinga2&epoch=ms&q=SHOW+TAG+VALUES++WITH+KEY+%3D+%22service%22+WHERE+hostname+%3D~+%2F%5Esems.vms.idial.fh%24%2F" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.791878879+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:datasources:query scopes:" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=query_data t=2024-07-17T13:52:18.792693548+02:00 level=debug msg="Processed metrics query" ref_id=metricFindQuery from=0 to=0 interval=1000 max_data_points=1000 query="{\"adhocFilters\":[],\"alias\":\"\",\"datasource\":{\"type\":\"influxdb\",\"uid\":\"000000001\"},\"datasourceId\":1,\"limit\":\"\",\"maxDataPoints\":1000,\"measurement\":\"\",\"policy\":\"\",\"query\":\"SHOW TAG VALUES WITH KEY = \\\"service\\\" WHERE hostname =~ /^sems.vms.idial.fh$/ AND service =~ /^apt$/\",\"rawQuery\":true,\"rawSql\":\"\",\"refId\":\"metricFindQuery\",\"slimit\":\"\",\"tz\":\"\"}" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=secrets.kvstore t=2024-07-17T13:52:18.792918143+02:00 level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=influxdb Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influxdb endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.793153368+02:00 level=debug msg="Received a query request" numQueries=1 Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influxdb endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.793367353+02:00 level=debug msg="Making a InfluxQL type query" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.793584698+02:00 level=debug msg="Influxdb query" rawquery="SHOW TAG VALUES WITH KEY = \"service\" WHERE hostname =~ /^sems.vms.idial.fh$/ AND service =~ /^apt$/" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.793751272+02:00 level=debug msg="Influxdb request" url="http://localhost:8086/query?db=icinga2&epoch=ms&q=SHOW+TAG+VALUES+WITH+KEY+%3D+%22service%22+WHERE+hostname+%3D~+%2F%5Esems.vms.idial.fh%24%2F+AND+service+%3D~+%2F%5Eapt%24%2F" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.807897868+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:alert.rules:read scopes:" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=live t=2024-07-17T13:52:18.817583011+02:00 level=debug msg="Client wants to subscribe" user=4 client=6922e0bf-02d7-47e5-9851-d597c6123cfc channel=1/grafana/dashboard/uid/Pcey9Axiz Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=live t=2024-07-17T13:52:18.817878338+02:00 level=debug msg="Found cached channel handler" channel=grafana/dashboard/uid/Pcey9Axiz Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.818730938+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:dashboards:read scopes:dashboards:uid:Pcey9Axiz" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=live t=2024-07-17T13:52:18.818885141+02:00 level=debug msg="Client subscribed" user=4 client=6922e0bf-02d7-47e5-9851-d597c6123cfc channel=1/grafana/dashboard/uid/Pcey9Axiz Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.851467362+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:datasources:query scopes:" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=query_data t=2024-07-17T13:52:18.851979673+02:00 level=debug msg="Processed metrics query" ref_id=A from=1721195538838 to=1721217138838 interval=30000 max_data_points=640 query="{\"adhocFilters\":[],\"alias\":\"$tag_metric\",\"datasource\":{\"type\":\"influxdb\",\"uid\":\"000000001\"},\"datasourceId\":1,\"dsType\":\"influxdb\",\"groupBy\":[{\"params\":[\"$__interval\"],\"type\":\"time\"},{\"params\":[\"metric\"],\"type\":\"tag\"},{\"params\":[\"none\"],\"type\":\"fill\"}],\"intervalMs\":30000,\"limit\":\"\",\"maxDataPoints\":640,\"measurement\":\"/^apt$/\",\"orderByTime\":\"ASC\",\"policy\":\"default\",\"query\":\"SELECT mean(\\\"value\\\") FROM /^apt$/ WHERE (\\\"hostname\\\" =~ /^sems\\\\.vms\\\\.idial\\\\.fh$/ AND \\\"service\\\" =~ /^apt$/) AND $timeFilter GROUP BY time($__interval), \\\"metric\\\" fill(none)\",\"rawQuery\":false,\"rawSql\":\"\",\"refId\":\"A\",\"resultFormat\":\"time_series\",\"select\":[[{\"params\":[\"value\"],\"type\":\"field\"},{\"params\":[],\"type\":\"mean\"}]],\"slimit\":\"\",\"tags\":[{\"key\":\"hostname\",\"operator\":\"=~\",\"value\":\"/^sems\\\\.vms\\\\.idial\\\\.fh$/\"},{\"condition\":\"AND\",\"key\":\"service\",\"operator\":\"=~\",\"value\":\"/^apt$/\"}],\"tz\":\"\"}" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=query_data t=2024-07-17T13:52:18.852212518+02:00 level=debug msg="Processed metrics query" ref_id=B from=1721195538838 to=1721217138838 interval=30000 max_data_points=640 query="{\"adhocFilters\":[],\"alias\":\"critical\",\"datasource\":{\"type\":\"influxdb\",\"uid\":\"000000001\"},\"datasourceId\":1,\"groupBy\":[{\"params\":[\"$__interval\"],\"type\":\"time\"},{\"params\":[\"previous\"],\"type\":\"fill\"}],\"hide\":false,\"intervalMs\":30000,\"limit\":\"\",\"maxDataPoints\":640,\"measurement\":\"/^apt$/\",\"orderByTime\":\"ASC\",\"policy\":\"default\",\"query\":\"\",\"rawSql\":\"\",\"refId\":\"B\",\"resultFormat\":\"time_series\",\"select\":[[{\"params\":[\"crit\"],\"type\":\"field\"},{\"params\":[],\"type\":\"last\"}]],\"slimit\":\"\",\"tags\":[{\"key\":\"hostname\",\"operator\":\"=~\",\"value\":\"/^sems\\\\.vms\\\\.idial\\\\.fh$/\"},{\"condition\":\"AND\",\"key\":\"service\",\"operator\":\"=~\",\"value\":\"/^apt$/\"}],\"tz\":\"\"}" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=query_data t=2024-07-17T13:52:18.852404183+02:00 level=debug msg="Processed metrics query" ref_id=C from=1721195538838 to=1721217138838 interval=30000 max_data_points=640 query="{\"adhocFilters\":[],\"alias\":\"warning\",\"datasource\":{\"type\":\"influxdb\",\"uid\":\"000000001\"},\"datasourceId\":1,\"groupBy\":[{\"params\":[\"$__interval\"],\"type\":\"time\"},{\"params\":[\"previous\"],\"type\":\"fill\"}],\"hide\":false,\"intervalMs\":30000,\"limit\":\"\",\"maxDataPoints\":640,\"measurement\":\"/^apt$/\",\"orderByTime\":\"ASC\",\"policy\":\"default\",\"query\":\"\",\"rawSql\":\"\",\"refId\":\"C\",\"resultFormat\":\"time_series\",\"select\":[[{\"params\":[\"warn\"],\"type\":\"field\"},{\"params\":[],\"type\":\"last\"}]],\"slimit\":\"\",\"tags\":[{\"key\":\"hostname\",\"operator\":\"=~\",\"value\":\"/^sems\\\\.vms\\\\.idial\\\\.fh$/\"},{\"condition\":\"AND\",\"key\":\"service\",\"operator\":\"=~\",\"value\":\"/^apt$/\"}],\"tz\":\"\"}" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=secrets.kvstore t=2024-07-17T13:52:18.852616288+02:00 level=debug msg="got secret value from cache" orgId=1 type=datasource namespace=influxdb Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influxdb endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.852917135+02:00 level=debug msg="Received a query request" numQueries=3 Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influxdb endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.853087508+02:00 level=debug msg="Making a InfluxQL type query" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.853330274+02:00 level=debug msg="Influxdb query" rawquery="SELECT mean(\"value\") FROM /^apt$/ WHERE (\"hostname\" =~ /^sems\\.vms\\.idial\\.fh$/ AND \"service\" =~ /^apt$/) AND time >= 1721195538838ms and time <= 1721217138838ms GROUP BY time(30s), \"metric\" fill(none) ORDER BY time ASC" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.853498348+02:00 level=debug msg="Influxdb request" url="http://localhost:8086/query?db=icinga2&epoch=ms&q=SELECT+mean%28%22value%22%29+FROM+%2F%5Eapt%24%2F+WHERE+%28%22hostname%22+%3D~+%2F%5Esems%5C.vms%5C.idial%5C.fh%24%2F+AND+%22service%22+%3D~+%2F%5Eapt%24%2F%29+AND+time+%3E%3D+1721195538838ms+and+time+%3C%3D+1721217138838ms+GROUP+BY+time%2830s%29%2C+%22metric%22+fill%28none%29+ORDER+BY+time+ASC" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.856684542+02:00 level=debug msg="Influxdb query" rawquery="SELECT last(\"crit\") FROM /^apt$/ WHERE (\"hostname\" =~ /^sems\\.vms\\.idial\\.fh$/ AND \"service\" =~ /^apt$/) AND time >= 1721195538838ms and time <= 1721217138838ms GROUP BY time(30s) fill(previous) ORDER BY time ASC" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.856851465+02:00 level=debug msg="Influxdb request" url="http://localhost:8086/query?db=icinga2&epoch=ms&q=SELECT+last%28%22crit%22%29+FROM+%2F%5Eapt%24%2F+WHERE+%28%22hostname%22+%3D~+%2F%5Esems%5C.vms%5C.idial%5C.fh%24%2F+AND+%22service%22+%3D~+%2F%5Eapt%24%2F%29+AND+time+%3E%3D+1721195538838ms+and+time+%3C%3D+1721217138838ms+GROUP+BY+time%2830s%29+fill%28previous%29+ORDER+BY+time+ASC" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.858069683+02:00 level=debug msg="Influxdb query" rawquery="SELECT last(\"warn\") FROM /^apt$/ WHERE (\"hostname\" =~ /^sems\\.vms\\.idial\\.fh$/ AND \"service\" =~ /^apt$/) AND time >= 1721195538838ms and time <= 1721217138838ms GROUP BY time(30s) fill(previous) ORDER BY time ASC" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=tsdb.influx_influxql endpoint=queryData pluginId=influxdb dsName=influxdb dsUID=000000001 uname=sa-1-autogen-1-icinga2 t=2024-07-17T13:52:18.858242407+02:00 level=debug msg="Influxdb request" url="http://localhost:8086/query?db=icinga2&epoch=ms&q=SELECT+last%28%22warn%22%29+FROM+%2F%5Eapt%24%2F+WHERE+%28%22hostname%22+%3D~+%2F%5Esems%5C.vms%5C.idial%5C.fh%24%2F+AND+%22service%22+%3D~+%2F%5Eapt%24%2F%29+AND+time+%3E%3D+1721195538838ms+and+time+%3C%3D+1721217138838ms+GROUP+BY+time%2830s%29+fill%28previous%29+ORDER+BY+time+ASC" Jul 17 13:52:18 icinga.vms.idial.fh grafana[3530418]: logger=accesscontrol t=2024-07-17T13:52:18.898834302+02:00 level=debug msg="Evaluating permissions" namespace=user id=4 orgID=1 permissions="action:annotations:read scopes:" Jul 17 13:52:19 icinga.vms.idial.fh grafana[3530418]: logger=live t=2024-07-17T13:52:19.612886566+02:00 level=debug msg="Client disconnected" user=4 client=6922e0bf-02d7-47e5-9851-d597c6123cfc reason="connection closed" elapsed=1.269407763s ```

SebastianOpeni commented 1 month ago

The same thing is happening for us: OS: debian 12 Grafana Version: 11.0.1

telution commented 1 month ago

Same issue. Downgraded from 3.11.1 to 3.11.0 then it worked....

tom-ch1 commented 1 month ago

also happens to my environment (debian 12, grafana 11.1.0, grafana-image-renderer 3.11.1). the relevant log entries:

logger=plugin.grafana-image-renderer t=2024-07-24T17:25:29.526799745+02:00 level=debug msg="Browser initialized" config="map[args:[--no-sandbox --disable-gpu] chromeBin:/var/lib/grafana/plugins/grafana-image-renderer/chrome/linux-128.0.6597.0/chrome-linux64/chrome clustering:map[maxConcurrency:5 mode:browser monitor:false timeout:30] deviceScaleFactor:1 dumpio:false emulateNetworkConditions:false headed:false height:500 ignoresHttpsErrors:false maxDeviceScaleFactor:4 maxHeight:3000 maxWidth:3000 mode:default pageZoomLevel:1 timingMetrics:false verboseLogging:true width:1000]" logger=plugin.grafana-image-renderer t=2024-07-24T17:25:29.572094052+02:00 level=error msg="Failed to get browser version" err=map[] logger=plugin.grafana-image-renderer t=2024-07-24T17:25:29.574639913+02:00 level=debug msg="using plugin" version=2

dependencies are all installed. Output of ldd shows no missing libraries:

ldd /var/lib/grafana/plugins/grafana-image-renderer/chrome/linux-128.0.6597.0/chrome-linux64/chrome

tom-ch1 commented 1 month ago

I can confirm that the issue was introduced in 3.11.1. Downgrading to 3.11.0 resolves it.

vincenttt7 commented 1 month ago

Same issue for me, Even if manual install the libX..... My OS is Ubuntu 22.04 Jammy with Grafana v9.5.2. Downgrade to grafana-image-renderer-3.11.0 is work for me. I use manual download zip way to install grafana-image-renderer due to private network.

grafana-cli --pluginUrl /var/lib/grafana/plugins/grafana-image-renderer-3.11.0.linux-amd64.zip plugins install grafana-image-renderer

dhoward-medallia commented 1 month ago

We had this same issue and reverted and all is working again.

AgnesToulet commented 1 month ago

Hi! Sorry for the delay, I wasn't able to reproduce the issue both on Ubuntu 20.04 and Ubuntu 22.04 (through WSL though).

@koelle25 From the following log:

logger=plugin.grafana-image-renderer t=2024-07-17T13:39:19.280660299+02:00 level=debug msg="chrome_crashpad_handler: --database is required"

Have you tried to set this in your Grafana ini file:

[plugin.grafana-image-renderer]
rendering_args = --no-sandbox,--disable-gpu,--database

You could maybe also try this: https://github.com/puppeteer/puppeteer/issues/11023#issuecomment-1776247197

koelle25 commented 1 month ago

Hi Agnes! Thanks for responding. Unfortunately the following did not change anything:

[plugin.grafana-image-renderer]
rendering_args = --no-sandbox,--disable-gpu,--database

But I wonder if this is a valid setting, as the docs don't mention it.

The trick from https://github.com/puppeteer/puppeteer/issues/11023#issuecomment-1776247197 actually worked. I set the variables in my /etc/default/grafana-server, then restarted grafana. Rendering started worked again.

My /etc/default/grafana-server

``` GRAFANA_USER=grafana GRAFANA_GROUP=grafana GRAFANA_HOME=/usr/share/grafana LOG_DIR=/var/log/grafana DATA_DIR=/var/lib/grafana MAX_OPEN_FILES=10000 CONF_DIR=/etc/grafana CONF_FILE=/etc/grafana/grafana.ini RESTART_ON_UPGRADE=true PLUGINS_DIR=/var/lib/grafana/plugins PROVISIONING_CFG_DIR=/etc/grafana/provisioning # Only used on systemd systems PID_FILE_DIR=/run/grafana # TEST Workaround for grafana-image-renderer-3.11.1 XDG_CONFIG_HOME=/tmp/.chromium XDG_CACHE_HOME=/tmp/.chromium ```

I then also tried using a different directory, i.e. inside the home directory of the grafana user (/usr/share/grafana/):

XDG_CONFIG_HOME=/usr/share/grafana/.config/chromium
XDG_CACHE_HOME=/usr/share/grafana/.cache/chromium

But then the rendering stopped working again. Even after pre-creating the directories, doing a chown grafana:grafana and even chmod 777 on them these did not work.

Dou you have any idead what has changed between 3.11.0 and 3.11.1 that could cause this behaviour?

AgnesToulet commented 1 month ago

Glad to hear the env variables are working!

Every time we make a release, the Chrome executable packaged with the image renderer is updated, so it probably comes from this update. Hopefully, the next release with the next Chrome update will fix it.

To test this theory, you can use the old version of Chrome (in the image renderer 3.11.0 package) with this setting:

[plugin.grafana-image-renderer]
rendering_chrome_bin = /custom/path/to/chrome

But the env variables solution is a better one as it's usually best to use the latest versions of software.

koelle25 commented 1 month ago

Hi Agnes,

thanks for the tip. I commented out the env variables in /etc/default/grafana, then used this setting in /etc/grafana/grafana.ini:

[plugin.grafana-image-renderer]
rendering_chrome_bin = /storage/grafana/grafana-image-renderer_3.11.0/chrome/linux-128.0.6534.0/chrome-linux64/chrome

That also works.

I also tried it with the newest release 3.11.2 - but this one also does not work out of the box, only if using the env variables or old Chrome. So the newly bundled Chrome unfortunately does not fix the problem.

3.11.0 contains Chrome 128.0.6534.0 -> works
3.11.1 contains Chrome 128.0.6597.0 -> does NOT work
3.11.2 contains Chrome 129.0.6644.0 -> does NOT work

I also found out that the packaged Chrome in 3.11.1 and 3.11.2 had extra files: deb.deps and rpm.deps. I checked my installed packages against the list in deb.deps and installed the missing packages (in my case fonts-liberation, libvulkan1 and xdg-utils). Then restarted the grafana-server.service, but it still didn't work.

koelle25 commented 3 weeks ago

Same problem with release 3.11.3, containing Chrome 129.0.6654.0. Still one of the workarounds (using XDG_* env variables or older Chrome 128.0.6534) is needed.

jkroepke commented 5 days ago

Hi, just using the offical docker.io/grafana/grafana-image-renderer image and the error occours.

koelle25 commented 3 days ago

Same problem with release 3.11.3, containing Chrome 129.0.6654.0. Still one of the workarounds (using XDG_* env variables or older Chrome 128.0.6534) is needed.

Still an issue in 3.11.4 (containing Chrome 130.0.6693.0).

dkoneill commented 22 hours ago

3.11.0 contains Chrome 128.0.6534.0 -> works

3.11.1 contains Chrome 128.0.6597.0 -> does NOT work

3.11.2 contains Chrome 129.0.6644.0 -> does NOT work

Confirming that downgrading to 3.11.0 works. Here is a command line method to install a particular version of the plugin:

grafana-cli plugins uninstall grafana-image-renderer
grafana-cli plugins install grafana-image-renderer 3.11.0
service grafana-server restart

grafana / grafana-image-renderer

[3.11.1+] rendering failed: Error: Failed to launch the browser process #545

grafana-cli --pluginUrl /var/lib/grafana/plugins/grafana-image-renderer-3.11.0.linux-amd64.zip plugins install grafana-image-renderer