Closed SvenFlock closed 3 years ago
jens-maus
commented
3 years ago Die Bedingung für den HeathCheck is wie folgt: https://github.com/jens-maus/RaspberryMatic/blob/master/buildroot-external/board/oci/Dockerfile#L22
D.h. bitte einfach mal während des Laufens von RaspberryMatic die Ausgaben von monit status und monit report initialising und monit report down präsentieren/analysieren.
SvenFlock
commented
3 years ago Sorry für die späte Antwort. Ich war geschäftlich verreist und konnte mich nicht einloggen wegen Zweifaktorauthenitifzierung. Nach den 11 Tagen ist der Container auf einmal healthy, von selbst. Ich starte jetzt mehrmals den Container neu und beobachte das Verhalten und berichte. Vielen Dank für die Befehle für den Healthcheck.
SvenFlock
commented
3 years ago So, das Verhalten ist konstant, hab es mehrmals ausprobiert. Das Problem scheint monit report initialising zu sein, das liefert 1 zurück. Irgendwann nach ein paar Stunden ist dann der Health-Check in Ordnung.
/ # monit status
Monit 5.27.2 uptime: 9m
Process 'hss_led'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 99
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 1
children 0
cpu 0.0%
cpu total 0.0%
memory 0.1% [2.9 MB]
memory total 0.1% [2.9 MB]
security attribute -
filedescriptors 4 [0.0% of 1048576 limit]
total filedescriptors 4
read bytes 74.3 B/s [84.5 kB total]
disk read bytes 0 B/s [1.1 MB total]
disk read operations 3.4 reads/s [2289 reads total]
write bytes 0 B/s [3.9 kB total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [4 writes total]
data collected Sat, 23 Oct 2021 09:38:47
Process 'syslogd'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 105
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 1
children 0
cpu 0.0%
cpu total 0.0%
memory 0.0% [652 kB]
memory total 0.0% [652 kB]
security attribute -
filedescriptors 4 [0.0% of 1048576 limit]
total filedescriptors 4
read bytes 0 B/s [265.0 kB total]
disk read bytes 0 B/s [0 B total]
disk read operations 0.0 reads/s [1522 reads total]
write bytes 0 B/s [277.5 kB total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [1514 writes total]
data collected Sat, 23 Oct 2021 09:38:44
Process 'udevd'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 111
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 1
children 0
cpu 0.0%
cpu total 0.0%
memory 0.1% [2.6 MB]
memory total 0.1% [2.6 MB]
security attribute -
filedescriptors 12 [0.0% of 1048576 limit]
total filedescriptors 12
read bytes 0 B/s [2.7 MB total]
disk read bytes 0 B/s [2.5 MB total]
disk read operations 0.0 reads/s [2454 reads total]
write bytes 0 B/s [9.0 kB total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [764 writes total]
data collected Sat, 23 Oct 2021 09:38:47
Program 'udevdEnabled'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 1
last output -
data collected Sat, 23 Oct 2021 09:38:44
Process 'irqbalance'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 144
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 2
children 0
cpu 0.0%
cpu total 0.0%
memory 0.0% [1.7 MB]
memory total 0.0% [1.7 MB]
security attribute -
filedescriptors 6 [0.0% of 1048576 limit]
total filedescriptors 6
read bytes 288.4 B/s [144.8 kB total]
disk read bytes 0 B/s [68 kB total]
disk read operations 1.8 reads/s [919 reads total]
write bytes 0 B/s [40 B total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [6 writes total]
data collected Sat, 23 Oct 2021 09:38:47
Program 'irqbalanceEnabled'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 0
last output -
data collected Sat, 23 Oct 2021 09:38:44
Process 'eq3configd'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 401
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 1
children 0
cpu 0.0%
cpu total 0.0%
memory 0.0% [1.3 MB]
memory total 0.0% [1.3 MB]
security attribute -
filedescriptors 4 [0.0% of 1048576 limit]
total filedescriptors 4
read bytes 17.5 B/s [14.8 kB total]
disk read bytes 0 B/s [48 kB total]
disk read operations 0.2 reads/s [134 reads total]
write bytes 0 B/s [4 B total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [1 writes total]
port response time 0.447 ms to localhost:43439 type UDP/IP protocol DEFAULT
data collected Sat, 23 Oct 2021 09:38:44
Process 'lighttpd'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 406
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 1
children 1
cpu 0.0%
cpu total 0.3%
memory 0.0% [468 kB]
memory total 0.2% [7.7 MB]
security attribute -
filedescriptors 3 [0.0% of 1048576 limit]
total filedescriptors 42
read bytes 0 B/s [1.3 kB total]
disk read bytes 0 B/s [12 kB total]
disk read operations 0.0 reads/s [6 reads total]
write bytes 0 B/s [4 B total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [1 writes total]
port response time 0.352 ms to localhost:49292 type TCP/IP protocol DEFAULT
port response time 0.295 ms to localhost:48181 type TCP/IP protocol DEFAULT
port response time 0.271 ms to localhost:42010 type TCP/IP protocol DEFAULT
port response time 0.249 ms to localhost:42001 type TCP/IP protocol DEFAULT
port response time 0.249 ms to localhost:42000 type TCP/IP protocol DEFAULT
port response time 0.249 ms to localhost:41999 type TCP/IP protocol DEFAULT
port response time 0.222 ms to localhost:9292 type TCP/IP protocol DEFAULT
port response time 0.213 ms to localhost:8181 type TCP/IP protocol DEFAULT
port response time 0.217 ms to localhost:2010 type TCP/IP protocol DEFAULT
port response time 0.217 ms to localhost:2001 type TCP/IP protocol DEFAULT
port response time 0.213 ms to localhost:2000 type TCP/IP protocol DEFAULT
port response time 0.214 ms to localhost:1999 type TCP/IP protocol DEFAULT
port response time 0.216 ms to localhost:443 type TCP/IP protocol DEFAULT
port response time 0.224 ms to localhost:80 type TCP/IP protocol DEFAULT
port response time 12.273 ms to localhost:443 type TCP/IP using TLS (certificate valid for 3358 days) protocol DEFAULT
data collected Sat, 23 Oct 2021 09:38:47
Process 'ssdpd'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 412
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 1
children 0
cpu 0.0%
cpu total 0.0%
memory 0.0% [1.2 MB]
memory total 0.0% [1.2 MB]
security attribute -
filedescriptors 4 [0.0% of 1048576 limit]
total filedescriptors 4
read bytes 0 B/s [3.3 kB total]
disk read bytes 0 B/s [24 kB total]
disk read operations 0.0 reads/s [12 reads total]
write bytes 0 B/s [4 B total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [1 writes total]
port response time 1.201 s to localhost:1900 type UDP/IP protocol DEFAULT
data collected Sat, 23 Oct 2021 09:38:46
Process 'sshd'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 421
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 1
children 0
cpu 0.0%
cpu total 0.0%
memory 0.1% [3.6 MB]
memory total 0.1% [3.6 MB]
security attribute -
filedescriptors 5 [0.0% of 1048576 limit]
total filedescriptors 5
read bytes 0 B/s [21.6 kB total]
disk read bytes 0 B/s [896 kB total]
disk read operations 0.0 reads/s [39 reads total]
write bytes 0 B/s [10 B total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [2 writes total]
data collected Sat, 23 Oct 2021 09:38:47
Program 'sshdEnabled'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 0
last output -
data collected Sat, 23 Oct 2021 09:38:46
Process 'hs485d'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:17
Program 'hs485dEnabled'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:33
Process 'multimacd'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 468
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 8
children 0
cpu 0.0%
cpu total 0.0%
memory 0.0% [1.3 MB]
memory total 0.0% [1.3 MB]
security attribute -
filedescriptors 7 [0.0% of 1048576 limit]
total filedescriptors 7
read bytes 17.1 B/s [14.1 kB total]
disk read bytes 0 B/s [196 kB total]
disk read operations 4.5 reads/s [2612 reads total]
write bytes 16.6 B/s [9.1 kB total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.9 writes/s [432 writes total]
data collected Sat, 23 Oct 2021 09:38:47
Program 'multimacdEnabled'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 0
last output -
data collected Sat, 23 Oct 2021 09:38:46
Process 'hmlangw'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:17
Program 'hmlangwEnabled'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:33
Program 'hmlangwDisabled'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 1
last output -
data collected Sat, 23 Oct 2021 09:38:46
Process 'rfd'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 498
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 5
children 0
cpu 0.0%
cpu total 0.0%
memory 0.3% [13.1 MB]
memory total 0.3% [13.1 MB]
security attribute -
filedescriptors 12 [0.0% of 1048576 limit]
total filedescriptors 12
read bytes 12.4 B/s [3.3 MB total]
disk read bytes 0 B/s [3.9 MB total]
disk read operations 1.0 reads/s [1288 reads total]
write bytes 0.4 B/s [827 B total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.4 writes/s [266 writes total]
port response time 0.482 ms to localhost:32001 type TCP/IP protocol DEFAULT
data collected Sat, 23 Oct 2021 09:38:47
Program 'rfdEnabled'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 0
last output -
data collected Sat, 23 Oct 2021 09:38:46
Process 'HMIPServer'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 523
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 58
children 0
cpu 0.3%
cpu total 0.3%
memory 2.4% [91.4 MB]
memory total 2.4% [91.4 MB]
security attribute -
filedescriptors 95 [0.0% of 1048576 limit]
total filedescriptors 95
read bytes 249.5 B/s [14.6 MB total]
disk read bytes 0 B/s [36.8 MB total]
disk read operations 6.9 reads/s [31773 reads total]
write bytes 226.1 B/s [1.1 MB total]
disk write bytes 0 B/s [12 kB total]
disk write operations 6.8 writes/s [4574 writes total]
port response time 1.521 ms to localhost:39292 type TCP/IP protocol DEFAULT
data collected Sat, 23 Oct 2021 09:38:46
Process 'ReGaHss'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 635
parent pid 1
uid 0
effective uid 0
gid 0
uptime 10m
threads 28
children 0
cpu 0.0%
cpu total 0.0%
memory 0.4% [16.1 MB]
memory total 0.4% [16.1 MB]
security attribute -
filedescriptors 10 [0.0% of 1048576 limit]
total filedescriptors 10
read bytes 1.8 kB/s [4.5 MB total]
disk read bytes 0 B/s [4.7 MB total]
disk read operations 1.1 reads/s [2143 reads total]
write bytes 0 B/s [365 B total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [9 writes total]
port response time 1.202 s to localhost:1998 type UDP/IP protocol DEFAULT
port response time 0.685 ms to localhost:31999 type TCP/IP protocol DEFAULT
port response time 2.250 ms to localhost:8183 type TCP/IP protocol HTTP
data collected Sat, 23 Oct 2021 09:38:47
Process 'crond'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
pid 856
parent pid 1
uid 0
effective uid 0
gid 0
uptime 9m
threads 1
children 0
cpu 0.0%
cpu total 0.0%
memory 0.0% [1.7 MB]
memory total 0.0% [1.7 MB]
security attribute -
filedescriptors 3 [0.0% of 1048576 limit]
total filedescriptors 3
read bytes 0 B/s [437.8 kB total]
disk read bytes 0 B/s [468 kB total]
disk read operations 0.0 reads/s [724 reads total]
write bytes 0 B/s [1.8 kB total]
disk write bytes 0 B/s [0 B total]
disk write operations 0.0 writes/s [51 writes total]
data collected Sat, 23 Oct 2021 09:38:47
System 'ccu'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
load average [0.38] [0.24] [0.18]
cpu 2.4%usr 1.5%sys 0.0%nice 0.0%iowait 0.0%hardirq 0.1%softirq 0.0%steal 0.0%guest 0.0%guestnice
memory usage 519.6 MB [13.5%]
swap usage 0 B [0.0%]
uptime 11m
boot time Sat, 23 Oct 2021 09:27:49
filedescriptors 1600 [0.0% of 2147483647 limit]
data collected Sat, 23 Oct 2021 09:38:47
Filesystem 'rootfs'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
filesystem type overlay
filesystem flags ro,relatime,lowerdir=/var/lib/docker/overlay2/l/XR7EDNCFKJRHH5CDZLJF5ZS5CP:/var/lib/docker/overlay2/l/GOCCJPOEEV4OY36TX3BU2XGVHV,upperdir=/var/lib/docker/overlay2/f7459894dd5adf60151c93cd371007da1020b4
permission 755
uid 0
gid 0
block size 4 kB
space total 29.0 GB (of which 4.2% is reserved for root user)
space free for non superuser 22.6 GB [77.9%]
space free total 23.8 GB [82.1%]
inodes total 1925760
inodes free 1774805 [92.2%]
data collected Sat, 23 Oct 2021 09:38:47
Filesystem 'userfs'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
filesystem type ext4
filesystem flags rw,noatime
permission 755
uid 0
gid 0
block size 4 kB
space total 29.0 GB (of which 4.2% is reserved for root user)
space free for non superuser 22.6 GB [77.9%]
space free total 23.8 GB [82.1%]
inodes total 1925760
inodes free 1774805 [92.2%]
data collected Sat, 23 Oct 2021 09:38:47
Filesystem 'usb1'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:30:21
Program 'hasUSB'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:30:36
Program 'temperature'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 10
last output 38.459
data collected Sat, 23 Oct 2021 09:38:47
Program 'internetCheck'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 0
last output -
data collected Sat, 23 Oct 2021 09:38:47
Program 'internetCheckEnabled'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 1
last output -
data collected Sat, 23 Oct 2021 09:38:47
Program 'uncleanShutdownCheck'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 1
last output -
data collected Sat, 23 Oct 2021 09:38:47
Program 'hb_rf_eth-Check'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:19
Program 'hb_rf_eth-CheckEnabled'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:34
Program 'rpi4usb3Check'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 0
last output ok
data collected Sat, 23 Oct 2021 09:38:47
Process 'tailscaled'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:17
Network 'tailscale0'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:17
Program 'tailscaleEnabled'
status Not monitored
monitoring status Not monitored
monitoring mode active
on reboot start
data collected Sat, 23 Oct 2021 09:29:34
Process 'redmatic-nodered'
status OK
monitoring status Monitored
monitoring mode passive
on reboot nostart
pid 1716
parent pid 1709
uid 0
effective uid 0
gid 0
uptime 5m
threads 11
children 0
cpu 0.1%
cpu total 0.1%
memory 3.5% [135.2 MB]
memory total 3.5% [135.2 MB]
security attribute -
filedescriptors 26 [1.3% of 2048 limit]
total filedescriptors 26
read bytes 214.7 B/s [47.3 MB total]
disk read bytes 0 B/s [4 kB total]
disk read operations 0.4 reads/s [6275 reads total]
write bytes 194.0 B/s [3.4 MB total]
disk write bytes 0 B/s [860 kB total]
disk write operations 0.3 writes/s [1680 writes total]
port response time 4.025 ms to localhost:1880/addons/red/ type TCP/IP protocol HTTP
data collected Sat, 23 Oct 2021 09:38:47
Program 'redmatic-diskusage'
status Waiting
monitoring status Waiting
monitoring mode passive
on reboot nostart
data collected Sat, 23 Oct 2021 09:38:47
/ # monit report initialising
1
/ # monit report down
0Dann ist meine Vermutung das das an den RedMatic Monit einträgen liegt die RedMatic selbst hinzufügt. Entfern die mal und schau was dann passiert. Weil man sieht ja in der liste das der RedMatic diskusage monit job noch bei "Waiting" hängt.
SvenFlock
commented
3 years ago Wenn ich den Punkt aus dem watchdog entferne, ist der Status ok, vielen Dank! Das löst den Health-Check. Ich verstehe dann noch nicht, warum es erst mit dem aktuellen Release zu diesem Verhalten gekommen ist. Daher kann das Issue gerne geschlossen werden.
Program 'redmatic-diskusage'
status Not monitored
monitoring status Not monitored
monitoring mode passive
on reboot nostart
data collected Sat, 23 Oct 2021 10:33:28
/ # monit report initialising
0
/ # Wenn das so ist müsstest du im RedMatic Projekt das melden/erfragen obs dafür ne lösung gibt.
Describe the bug Docker Container ccu Health Check zeigt nach letztem release (20211009) unhealty an. Auf den ersten Blick scheint alles zu funktionieren. Inspect liefert keine genauen Details:
Steps to reproduce the behavior
Expected behavior Nach erfolgreichem Start sollte der healty-check positiv sein und bleiben.
Screenshots Siehe Code-Snippet von der Konsole
System information:
Additional context Zeitgleich gab es über Debian aus dem pivccu Repo ein Update der Kernelmodule. Eventuell liegt es damit zusammen. Ich habe das Verhalten als Bug gekennzeichnet, da das Verhalten vor dem Update nicht vorhanden war, aber trotz "unhealthy" alles auf den ersten Blick funktioniert.
Auch ist direkt nach dem Start der Container „healthy“, nach 1-2 Minuten wechselt er in den Status „unhealthy“.