monitoring grafana fails to install

[redlingg]

Hello this is the error message after deployment: (cadvisor and node seem to be ok - Raspbian GNU/Linux 11 (bullseye) ) any idea?

Deployment error failed to deploy a stack: node-exporter Pulling prometheus Pulling cadvisor Pulling grafana Pulling 5480d2ca1740 Pulling fs layer 9acb2e5ab1b1 Pulling fs layer 92bb93a79041 Pulling fs layer 8b2610846236 Pulling fs layer 0b96fb83d2bc Pulling fs layer 92bb93a79041 Waiting 08e5be749a8c Pulling fs layer 81b22afe0b98 Pulling fs layer 08e5be749a8c Waiting 81b22afe0b98 Waiting 865f832891d7 Pulling fs layer 77e5648788cd Pulling fs layer aee1f7df2306 Pulling fs layer ebb74e959ef3 Pulling fs layer 7c2f9cdc70d3 Pulling fs layer 6850e5bcc373 Pulling fs layer 13378d02925f Pulling fs layer 0b96fb83d2bc Waiting 77e5648788cd Waiting 865f832891d7 Waiting ebb74e959ef3 Waiting 7c2f9cdc70d3 Waiting 6850e5bcc373 Waiting 13378d02925f Waiting e6f6854e4a39 Pulling fs layer 8cf9e3322930 Pulling fs layer aee1f7df2306 Waiting e6f6854e4a39 Waiting 0adef5ba36ca Pulling fs layer 8cf9e3322930 Waiting

[oijkn]

It seems that the problem is more on the raspberry side than the Docker. Have you checked that your raspberry is correctly connecting to Docker hub ?

[redlingg]

well...I guess yes. if the other containers are created and running according to portainer and grafana just created but cannot be started...the connection should be ok. When starting: request faild with status code 400...

[oijkn]

Can you provide me the details of the command :

docker --version docker-compose --version docker image ls docker ps -a

[redlingg]

Docker version 20.10.12, build e91ed57

docker-compose version 1.29.2, build unknown

REPOSITORY TAG IMAGE ID CREATED SIZE lscr.io/linuxserver/jackett arm32v7-latest 9d226f98550e 28 hours ago 222MB netdata/netdata latest ab71303ac418 32 hours ago 329MB lscr.io/linuxserver/transmission arm32v7-latest 77fd486068e0 4 days ago 88.3MB portainer/portainer-ce latest 4e28534eed58 4 weeks ago 245MB containrrr/watchtower latest 72324c978d08 7 weeks ago 14.5MB pihole/pihole 2022.01.1 42ba116abbdb 2 months ago 235MB emby/embyserver_arm32v7 latest 5b8eba8db0fa 3 months ago 229MB fjctp/armhf-motion latest ad0a096bc790 5 years ago 402MB

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ecbdca2f3938 lscr.io/linuxserver/jackett:arm32v7-latest "/init" 18 hours ago Up 13 hours 0.0.0.0:9117->9117/tcp, :::9117->9117/tcp jackett a3e8ad68370f netdata/netdata:latest "/usr/sbin/run.sh" 18 hours ago Up 13 hours (healthy) 0.0.0.0:19999->19999/tcp, :::19999->19999/tcp netdata 50d23a1bcde1 lscr.io/linuxserver/transmission:arm32v7-latest "/init" 2 days ago Up 13 hours 0.0.0.0:9091->9091/tcp, :::9091->9091/tcp, 0.0.0.0:51413->51413/tcp, :::51413->51413/tcp, 0.0.0.0:51413->51413/udp, :::51413->51413/udp transmission 343e6847fba7 pihole/pihole:2022.01.1 "/s6-init" 5 days ago Up 13 hours (healthy) 0.0.0.0:53->53/tcp, 0.0.0.0:53->53/udp, :::53->53/tcp, :::53->53/udp, 67/udp, 0.0.0.0:85->80/tcp, :::85->80/tcp pihole 0b72c59b8329 containrrr/watchtower:latest "/watchtower" 7 days ago Up 13 hours 8080/tcp watchtower 4f5005250b38 fjctp/armhf-motion:latest "/entrypoint.sh" 8 days ago Up 13 hours 0.0.0.0:9001->8081/tcp, :::9001->8081/tcp motion-server 06816f7c4cf1 emby/embyserver_arm32v7 "/init" 9 days ago Up 13 hours embyserver f70443d77a9d portainer/portainer-ce:latest "/portainer" 9 days ago Up 13 hours 8000/tcp, 9443/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp portainer

NETDATA WAS NOT INSTALLED WHEN THE PROBLEM OCCURED...

[Bu5terGit]

I have the same issue as above. followed as discribed and it fails on Grafana starting. Docker version 20.10.17, build 100c701 docker-compose version 1.29.2, build unknown

[ankitsinghaniyaz]

I'm facing the same issue on my raspberry pi, my error looks something like this

failed to deploy a stack: Network raspberrypi-docker-monitor_rpimonitor_default Creating Network raspberrypi-docker-monitor_rpimonitor_default Created Container monitoring-grafana Creating Container monitoring-node-exporter Creating Container monitoring-cadvisor Creating Container monitoring-grafana Created Container monitoring-node-exporter Created Container monitoring-cadvisor Created Container monitoring-prometheus Creating Container monitoring-prometheus Created Container monitoring-grafana Starting Container monitoring-node-exporter Starting Container monitoring-cadvisor Starting Container monitoring-node-exporter Started Container monitoring-cadvisor Started Container monitoring-prometheus Starting Container monitoring-prometheus Started Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/portainer/Files/AppData/Config/grafana/grafana.ini" to rootfs at "/etc/grafana/grafana.ini": mount /portainer/Files/AppData/Config/grafana/grafana.ini:/etc/grafana/grafana.ini (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type ```

[TimPasquini]

Edited: Same issue as @ankitsinghaniyaz

I've managed to straighten out some stuff and get this running. For anyone else having issues since the directions are a bit sparse... You'll need to manually create the following folders on your system somewhere:

/grafana/data /grafana/provisioning /grafana/grafana.ini /prometheus/

You can get a grafana.ini very easily by running the following with the grafana directory as your working directory: docker run --rm --entrypoint /bin/bash grafana/grafana:latest -c 'cat $GF_PATHS_CONFIG' > grafana.ini

Copy the provided prometheus.yml into your /prometheus/ folder

Open the provided docker-compose.yml and update the path information under grafana and prometheus. You may still have permissions issues at this point. The easiest (though probably not most secure) thing to do is to just set your grafana and prometheus folders permissions to anyone.

[oijkn]

I'm working on new version, that will fix this problem.

[oijkn]

Can you try my last update, check the README please.

[TimPasquini]

Ok, definitely went smoother. I still had failures due to permission issues and had to set permissions to Anyone to get it to work.

As a note, the dashboard link in the readme is truncated and 404s, it should be: https://grafana.com/grafana/dashboards/15120-raspberry-pi-docker-monitoring/

That wasn't actually an issue for me though, as the provisioning process you have set up seems to have autoloaded it correctly.

[oijkn]

Thanks for your remark about the link to the dashboard, I will correct it.

On the other hand I did not understand your problem with permission, can you give me more details ?

[TimPasquini]

Sorry for the delay. The grafana and prometheus containers keep restarting for me. When I check the logs in the containers they both return errors about not having permission to write to the volumes we created. I ended up using chmod to set the permission to anyone. I had followed the directions you wrote and can confirm that the users you defined in the chown commands were showing as the owner.

[oijkn]

Can you provide me some logs and show me the owner of prometheus/grafana directory in your host?

[TimPasquini]

I've got it working well enough for now, when I get a chance I'll set up a new RPiOS sdcard and capture it for you. If you don't want to wait, I was setting this up on a fresh Raspberry Pi 4b on the latest RPiOS. I followed the docker daemon group setup instructions here: https://docs.docker.com/engine/install/linux-postinstall/

[oijkn]

So if I understand correctly, the observability stack works well on your Raspberry ?

[oijkn]

This issue was closed because it has been open for 30 days with no activity.