kuhn-ruess / Checkmk-Checks

Checks and Stuff for Check_MK
MIT License
36 stars 20 forks source link

Docker check-mk 2.2.0p12 Parsing of section docker_containers failed #70

Closed Semetra22 closed 8 months ago

Semetra22 commented 8 months ago

The error message says Parsing of section docker_containers failed - please submit a crash report!

Unfortunately, I don't know exactly where to start to find the error. Perhaps someone here has an idea and could help me?

Traceback:

File "/omd/sites/monitoring/lib/python3/cmk/base/agent_based/checking/_checking.py", line 413, in get_aggregated_result
    consume_check_results(
  File "/omd/sites/monitoring/lib/python3/cmk/base/api/agent_based/checking_classes.py", line 484, in consume_check_results
    for subr in subresults:
  File "/omd/sites/monitoring/lib/python3/cmk/base/api/agent_based/register/check_plugins.py", line 93, in filtered_generator
    for element in generator(*args, **kwargs):
  File "/omd/sites/monitoring/local/lib/python3/cmk/base/plugins/agent_based/docker_images.py", line 63, in check_docker_images
    image_containers = get_running_image_containers(image["ImageID"], section_docker_containers.values())

Agent output:

<<<check_mk>>>
Version: 2.2.0p12
AgentOS: linux
Hostname: docker01
AgentDirectory: /etc/check_mk
DataDirectory: /var/lib/check_mk_agent
SpoolDirectory: /var/lib/check_mk_agent/spool
PluginsDirectory: /usr/lib/check_mk_agent/plugins
LocalDirectory: /usr/lib/check_mk_agent/local
FailedPythonReason: 
SSHClient: 
<<<checkmk_agent_plugins_lnx:sep(0)>>>
pluginsdir /usr/lib/check_mk_agent/plugins
localdir /usr/lib/check_mk_agent/local
/usr/lib/check_mk_agent/plugins/check_docker.py:CMK_VERSION="unversioned"

 -- all the other stuff -- 

<<<docker_info:sep(59)>>>
service;up
images;260
go_routines;295
file_descriptors;345
events_listeners;0
<<<docker_containers:sep(35)>>>
/docker_nginx#Names=/docker_nginx#State=running#Status=Up 5 days#Created=1699002787#Command=/docker-entrypoint.sh nginx -g 'daemon off;'#Image=nginx:stable#ImageID=sha256:967ca3be8ed4cdc3b937a7b5a5a899166389e2cf3704d9f588f4db903488d12f#SizeRootFs=142225623#SizeRw=1095#CPU_usage=15604690388.000000#CPU_system_usage=29042229190000000.000000#Memory_used=18673664#Memory_limit=26267291648
/docker_relution#Names=/docker_relution#State=running#Status=Up 5 days (healthy)#Created=1699002787#Command=/relution-docker.sh /bin/sh -c 'su relution --command /opt/relution/bin/relution-foreground.sh'#Image=relution/relution:latest#ImageID=sha256:f1f0d027784c421ad34c89db60f7230de80725fbe3af440a69f9985343806bfe#SizeRootFs=807987119#SizeRw=5765477#CPU_usage=1065086918993.000000#CPU_system_usage=29042237200000000.000000#Memory_used=3100794880#Memory_limit=26267291648
/docker_mariadb#Names=/docker_mariadb#State=running#Status=Up 5 days#Created=1699002786#Command=docker-entrypoint.sh mariadbd#Image=mariadb:latest#ImageID=sha256:f35870862d64d0e29598fba1d7f75cfefeb3f891cb22b3e2d4459c903e666554#SizeRootFs=403893627#SizeRw=2#CPU_usage=190649661627.000000#CPU_system_usage=29042245220000000.000000#Memory_used=475365376#Memory_limit=26267291648
/gitlab-runner-runner-1#Names=/gitlab-runner-runner-1#State=running#Status=Up 2 weeks#Created=1697796628#Command=/usr/bin/dumb-init /entrypoint run --user=gitlab-runner --working-directory=/home/gitlab-runner#Image=registry.gitlab.com/gitlab-org/gitlab-runner:alpine#ImageID=sha256:9c9c9166bad3a74c03739ccd3c1600d3d56eb2076c22fddbf20aeda76e42b0ff#SizeRootFs=362117596#CPU_usage=1008752266919.000000#CPU_system_usage=29042253250000000.000000#Memory_used=29958144#Memory_limit=26267291648
/gitlab-runner-dind-1#Names=/gitlab-runner-dind-1#State=running#Status=Up 2 weeks#Created=1697796628#Command=dockerd-entrypoint.sh --storage-driver=overlay2#Image=docker:19-dind#ImageID=sha256:fa3b7a5678ece1ea157bb3f4fb6372911ad054e9d271f09d6401fb53e6a8c97d#SizeRootFs=236507799#SizeRw=58804#CPU_usage=9240814336575.000000#CPU_system_usage=29042261270000000.000000#Memory_used=1188192256#Memory_limit=26267291648
/api-api-server-1#Names=/api-api-server-1#State=running#Status=Up 3 weeks#Created=1697185316#Command=/app/shell2http -no-index -form -basic-auth=upstream:tAYB3ftAnEX8fDF -basic-auth=ddapi:emilfrey -log=/home/scripts/api.log /v0/dealerdesk '/home/scripts/ddi2dealerdesk.sh $v_param $v_ddi $v_callerid' /v0/hassio /home/scripts/alamos.sh#Image=api-server#ImageID=sha256:3b1a706c0455d2a76ed662c6e48e4cd9c5d4dae4df2880907a124aa6d2f7f6e9#SizeRootFs=14830695#CPU_usage=1731363550.000000#CPU_system_usage=29042269260000000.000000#Memory_used=11706368#Memory_limit=26267291648
/smtp-relay3-postfix-1#Names=/smtp-relay3-postfix-1#State=running#Status=Up 4 weeks#Created=1696845202#Command=/run.sh#Image=juanluisbaptiste/postfix:latest#ImageID=sha256:208469f0fbb228924502c739eedacc8ba6b68df916a09e0fc083ce2f996719a0#SizeRootFs=50659838#SizeRw=53381#CPU_usage=20915115695.000000#CPU_system_usage=29042277330000000.000000#Memory_used=2523136#Memory_limit=26267291648
/mailu-webmail-1#Names=/mailu-webmail-1#State=running#Status=Up 5 weeks (healthy)#Created=1695989550#Command=docker-php-entrypoint /bin/sh -c /start.py#Image=mailu/roundcube:1.7#ImageID=sha256:9ca8810fbc8cd4b573b9f976073e2958d51eead18413296e2555dec86c04f8f6#SizeRootFs=605351090#SizeRw=2172#CPU_usage=6219979342294.000000#CPU_system_usage=29042285320000000.000000#Memory_used=53092352#Memory_limit=26267291648
/mailu-imap-1#Names=/mailu-imap-1#State=running#Status=Up 5 weeks (healthy)#Created=1695989550#Command=/bin/sh -c /start.py#Image=mailu/dovecot:1.7#ImageID=sha256:19e75fb28874f25327e97f9da75b842bb71b72f47abb27f5addf304b32ae3f69#SizeRootFs=167886836#SizeRw=35724#CPU_usage=5449763881731.000000#CPU_system_usage=29042293350000000.000000#Memory_used=30789632#Memory_limit=26267291648
/mailu-antispam-1#Names=/mailu-antispam-1#State=running#Status=Up 5 weeks (healthy)#Created=1695989550#Command=/bin/sh -c /start.py#Image=mailu/rspamd:1.7#ImageID=sha256:5adb1273b6848f41b22ea4af8301ac3a99e661ff3899d54fe4bd7f7909a8338f#SizeRootFs=138020436#SizeRw=2296#CPU_usage=4197214229157.000000#CPU_system_usage=29042301370000000.000000#Memory_used=81731584#Memory_limit=26267291648
/mailu-smtp-1#Names=/mailu-smtp-1#State=running#Status=Up 5 weeks (healthy)#Created=1695989550#Command=/bin/sh -c /start.py#Image=mailu/postfix:1.7#ImageID=sha256:47b018ea18d321c2b1541e0780ea5bbef91748d103b1338c395b21922b209be9#SizeRootFs=92794430#SizeRw=23574#CPU_usage=3427357395578.000000#CPU_system_usage=29042309380000000.000000#Memory_used=41758720#Memory_limit=26267291648
/mailu-admin-1#Names=/mailu-admin-1#State=running#Status=Up 5 weeks (healthy)#Created=1695989549#Command=/bin/sh -c /start.py#Image=mailu/admin:1.7#ImageID=sha256:80bdfd0088cc431c28443c5f209743e9dbe402b2cd50cebfb15ac5abcaed5b24#SizeRootFs=172771995#SizeRw=118081#CPU_usage=19543680259474.000000#CPU_system_usage=29042317390000000.000000#Memory_used=247582720#Memory_limit=26267291648
/mailu-front-1#Names=/mailu-front-1#State=running#Status=Up 5 weeks (healthy)#Created=1695989549#Command=/bin/sh -c /start.py#Image=mailu/nginx:1.7#ImageID=sha256:fc63a560ebccd6258eed155b3fdf759aaf253499d316a50c1c7b808eb83f0baa#SizeRootFs=103953389#SizeRw=5237#CPU_usage=3958026267128.000000#CPU_system_usage=29042325380000000.000000#Memory_used=10129408#Memory_limit=26267291648
/mailu-antivirus-1#Names=/mailu-antivirus-1#State=running#Status=Up 5 weeks (healthy)#Created=1695989549#Command=/bin/sh -c /start.py#Image=mailu/clamav:1.7#ImageID=sha256:3d56531dbd75e83f13cdefaf144ce4bb35640dd2046f4117a17515b06d792225#SizeRootFs=68480036#SizeRw=3#CPU_usage=3805285458342.000000#CPU_system_usage=29042333390000000.000000#Memory_used=1291476992#Memory_limit=26267291648
/mailu-redis-1#Names=/mailu-redis-1#State=running#Status=Up 5 weeks#Created=1695989549#Command=docker-entrypoint.sh redis-server#Image=redis:alpine#ImageID=sha256:34e1dc356a22aadc8d41f9335ad4b5fdcf9f4c1472df3523846ad2e73d7d6c41#SizeRootFs=32373327#CPU_usage=3484038353660.000000#CPU_system_usage=29042341390000000.000000#Memory_used=11132928#Memory_limit=26267291648
/mailu-fetchmail-1#Names=/mailu-fetchmail-1#State=running#Status=Up 5 weeks#Created=1695989549#Command=/fetchmail.py#Image=mailu/fetchmail:1.7#ImageID=sha256:21f2c9a8d190283c8e19fd9890ccf1a86a686298decac198ca6588bf0ac374a6#SizeRootFs=66469502#CPU_usage=19602150280.000000#CPU_system_usage=29042349420000000.000000#Memory_used=12783616#Memory_limit=26267291648
/mailu-webdav-1#Names=/mailu-webdav-1#State=running#Status=Up 5 weeks (healthy)#Created=1695989549#Command=/bin/sh -c 'radicale -f -S -C /radicale.conf'#Image=mailu/radicale:1.7#ImageID=sha256:e34e64c169ff9b9df3df2277d858fbc84ed55c81437e0d0a19964506915d221b#SizeRootFs=62009273#CPU_usage=4318436584385.000000#CPU_system_usage=29042357410000000.000000#Memory_used=882294784#Memory_limit=26267291648
/zerotier-one#Names=/zerotier-one#State=running#Status=Up 5 weeks#Created=1695987389#Command=entrypoint.sh -U#Image=zyclonite/zerotier#ImageID=sha256:95dc4ee46fe9992908c0e7c32f81f718046ff35d1c9234e0b214cbbe37c1ac0f#SizeRootFs=13758439#CPU_usage=2334059394440.000000#CPU_system_usage=29042365440000000.000000#Memory_used=35958784#Memory_limit=26267291648
/smtp-relay-postfix-1#Names=/smtp-relay-postfix-1#State=running#Status=Up 6 weeks#Created=1695823832#Command=/run.sh#Image=juanluisbaptiste/postfix:latest#ImageID=sha256:208469f0fbb228924502c739eedacc8ba6b68df916a09e0fc083ce2f996719a0#SizeRootFs=50646358#SizeRw=39901#CPU_usage=21235596573.000000#CPU_system_usage=29042373450000000.000000#Memory_used=2105344#Memory_limit=26267291648
/smtp-relay2-postfix-1#Names=/smtp-relay2-postfix-1#State=running#Status=Up 6 weeks#Created=1695823814#Command=/run.sh#Image=juanluisbaptiste/postfix:latest#ImageID=sha256:208469f0fbb228924502c739eedacc8ba6b68df916a09e0fc083ce2f996719a0#SizeRootFs=50646371#SizeRw=39914#CPU_usage=21325845856.000000#CPU_system_usage=29042381470000000.000000#Memory_used=2101248#Memory_limit=26267291648
/ipinfo.tw#Names=/ipinfo.tw#State=running#Status=Up 6 weeks (healthy)#Created=1695822772#Command=/usr/sbin/nginx -g 'daemon off;'#Image=peterdavehello/ipinfo.tw:latest#ImageID=sha256:69eb228a48dcdd19069afc0f1bdc84b013e61f118c0458f4df1022a1fd5e6137#SizeRootFs=30573679#SizeRw=9791950#CPU_usage=3220194451582.000000#CPU_system_usage=29042389450000000.000000#Memory_used=39616512#Memory_limit=26267291648
/ddapi-flask-api-1#Names=/ddapi-flask-api-1#State=running#Status=Up 6 weeks#Created=1695025139#Command=flask run --host=0.0.0.0#Image=ddapi-flask-api#ImageID=sha256:83f7aa794485cb64c44edcba8746c0635099b8da8875484b2a1a2f96c2de189f#SizeRootFs=1210781948#SizeRw=141148#CPU_usage=579422569427.000000#CPU_system_usage=29042397430000000.000000#Memory_used=147947520#Memory_limit=26267291648
/zu-main#Names=/zu-main#State=running#Status=Up 6 weeks#Created=1694616020#Command=docker-entrypoint.sh node ./bin/www#Image=dec0dos/zero-ui:latest#ImageID=sha256:6e69b96b7b499acd82a2d851991691960f67723da67974057172fb66893240cc#SizeRootFs=129512619#CPU_usage=566462087148.000000#CPU_system_usage=29042405450000000.000000#Memory_used=81993728#Memory_limit=26267291648
/zu-controller#Names=/zu-controller#State=running#Status=Up 6 weeks#Created=1694616020#Command=zerotier-one#Image=dec0dos/zerotier-controller:latest#ImageID=sha256:fc4a8493a1a85f1771f5e3d85229eb766cb772972e17ab3fd655b1ec04485767#SizeRootFs=10959476#CPU_usage=902622988901.000000#CPU_system_usage=29042413480000000.000000#Memory_used=290197504#Memory_limit=26267291648
/reverseproxy-app-1#Names=/reverseproxy-app-1#State=running#Status=Up 6 weeks#Created=1694097516#Command=/init#Image=jc21/nginx-proxy-manager:latest#ImageID=sha256:9c3f57826a5d0a82720533269d1996931d471f8130a0edb58d4a6602a0a13a8c#SizeRootFs=1261328528#SizeRw=418288880#CPU_usage=5461946764024.000000#CPU_system_usage=29042421480000000.000000#Memory_used=1658298368#Memory_limit=26267291648
/reverseproxy-db-1#Names=/reverseproxy-db-1#State=running#Status=Up 6 weeks#Created=1694097511#Command=/scripts/run.sh#Image=jc21/mariadb-aria:latest#ImageID=sha256:d3aa3f6e075ef47fb0c9919f73cd2f2c57010c47ca5dfc2b5b692930dc479fe6#SizeRootFs=217487539#SizeRw=101#CPU_usage=38045234191.000000#CPU_system_usage=29042429500000000.000000#Memory_used=76673024#Memory_limit=26267291648
/upsnap#Names=/upsnap#State=running#Status=Up 6 weeks (healthy)#Created=1692012761#Command=/bin/sh -c './upsnap serve --http 0.0.0.0:7777'#Image=ghcr.io/seriousm4x/upsnap:3#ImageID=sha256:6a53027301320256ac9baf41916077eb80bc325204f647f02d2670cc401684bb#SizeRootFs=116701508#SizeRw=8867#CPU_usage=9898422945621.000000#CPU_system_usage=29042437510000000.000000#Memory_used=51138560#Memory_limit=26267291648
/mailalert-fetchmail-1#Names=/mailalert-fetchmail-1#State=running#Status=Up 6 weeks#Created=1691738056#Command=/bin/sh /bin/start.sh#Image=cguenther/docker-fetchmail#ImageID=sha256:b14e05957575b08765316bf4c3790afc6bc3a47e902ec5168ffed328b015bc09#SizeRootFs=10293760#SizeRw=386#CPU_usage=106012865088.000000#CPU_system_usage=29042445530000000.000000#Memory_used=5009408#Memory_limit=26267291648
/snapdrop#Names=/snapdrop#State=running#Status=Up 6 weeks#Created=1687849669#Command=/init#Image=ghcr.io/linuxserver/snapdrop#ImageID=sha256:54825f0d4e1499ed8664c5fb332cb0a8736057752c0641786b115213007e002d#SizeRootFs=128373203#SizeRw=26503#CPU_usage=108035604343.000000#CPU_system_usage=29042453530000000.000000#Memory_used=99475456#Memory_limit=26267291648
/vaultwarden#Names=/vaultwarden#State=running#Status=Up 6 weeks (healthy)#Created=1687792010#Command=/start.sh#Image=vaultwarden/server:latest#ImageID=sha256:39c214581b301f67ed1697faef34b2e429f739be231508e1ade853e67c409378#SizeRootFs=204315699#CPU_usage=2359283125109.000000#CPU_system_usage=29042461560000000.000000#Memory_used=151793664#Memory_limit=26267291648
/lamp-php8#Names=/lamp-php8#State=running#Status=Up 6 weeks#Created=1686126822#Command=docker-php-entrypoint apache2-foreground#Image=lamp_webserver#ImageID=sha256:ac530d7fc78e2ffbbb9dc18ae47110831220eec33cb60b04c67dff2c1271ae04#SizeRootFs=769941932#SizeRw=2#CPU_usage=84374326269.000000#CPU_system_usage=29042469580000000.000000#Memory_used=60825600#Memory_limit=26267291648
/lamp-mysql8#Names=/lamp-mysql8#State=running#Status=Up 6 weeks#Created=1686126822#Command=docker-entrypoint.sh mysqld#Image=lamp_database#ImageID=sha256:e28f19f08742938264c7c677d3c138b3d7bd43cab09e8b22921a86c65b6babcd#SizeRootFs=564658837#SizeRw=6#CPU_usage=7974374138472.000000#CPU_system_usage=29042477620000000.000000#Memory_used=487923712#Memory_limit=26267291648
/shlink-web-client#Names=/shlink-web-client#State=running#Status=Up 6 weeks#Created=1685001208#Command=/docker-entrypoint.sh nginx -g 'daemon off;'#Image=shlinkio/shlink-web-client#ImageID=sha256:d3687046afb0a0e7e63cf50642a5c4c021f25d64ad9c9b500d54049a24583023#SizeRootFs=32195821#SizeRw=1446#CPU_usage=85961746.000000#CPU_system_usage=29042485630000000.000000#Memory_used=13594624#Memory_limit=26267291648
/kms_kms-server_1#Names=/kms_kms-server_1#State=running#Status=Up 6 weeks#Created=1685000835#Command=/bin/sh -c 'vlmcsdmulti-x64-musl-static vlmcsd -L 0.0.0.0:1688 -e -D'#Image=luodaoyi/kms-server:latest#ImageID=sha256:3ed72d1427841db4fa8f3db2560bdb57f3ed9bc98995e3660ce7d4d4dfa997ef#SizeRootFs=5744037#CPU_usage=162488818.000000#CPU_system_usage=29042493620000000.000000#Memory_used=1720320#Memory_limit=26267291648
/netmaker-ui#Names=/netmaker-ui#State=running#Status=Up 6 weeks#Created=1674550043#Command=/docker-entrypoint.sh#Image=gravitl/netmaker-ui:v0.17.1#ImageID=sha256:d375bd1f3ae1ef87d5ed8e82ff1b0c4928eff48b58bcd23c937e94dec893647c#SizeRootFs=32452722#SizeRw=64#CPU_usage=56364971.000000#CPU_system_usage=29042501640000000.000000#Memory_used=9990144#Memory_limit=26267291648
/coredns#Names=/coredns#State=running#Status=Up 6 weeks#Created=1674550043#Command=/coredns -conf /root/dnsconfig/Corefile#Image=coredns/coredns#ImageID=sha256:0ad7f9e4a77d03111e29c93cd609a7fc9f45cde4ea01c7d25319d69d1b1fafb5#SizeRootFs=50249337#CPU_usage=504969938339.000000#CPU_system_usage=29042509670000000.000000#Memory_used=62418944#Memory_limit=26267291648
/netmaker#Names=/netmaker#State=running#Status=Up 10 seconds#Created=1674550033#Command=./netmaker#Image=gravitl/netmaker:v0.17.1#ImageID=sha256:b57abf1fb2720943079219d48242afc16dc881a8a9c3812a44e66a8d71afb3c2#SizeRootFs=29006476#CPU_usage=89568856.000000#CPU_system_usage=29042517690000000.000000#Memory_used=10330112#Memory_limit=26267291648
/bookstack#Names=/bookstack#State=running#Status=Up 6 weeks#Created=1669805007#Command=/init#Image=lscr.io/linuxserver/bookstack#ImageID=sha256:8b90d6b4e707d08f23e26e4370c46151fa5213823f3fd11baec351b2c4109340#SizeRootFs=383207556#SizeRw=121348936#CPU_usage=339110283521.000000#CPU_system_usage=29042525720000000.000000#Memory_used=121495552#Memory_limit=26267291648
/bookstack_db#Names=/bookstack_db#State=running#Status=Up 6 weeks#Created=1669805004#Command=/init#Image=lscr.io/linuxserver/mariadb#ImageID=sha256:131ba706f873cfd651f9c6cca357833afa60a0847c735b86074c8aab03cbe039#SizeRootFs=289206401#SizeRw=28530#CPU_usage=317454475828.000000#CPU_system_usage=29042533750000000.000000#Memory_used=129323008#Memory_limit=26267291648
/ddns_ddns_1#Names=/ddns_ddns_1#State=running#Status=Up 6 weeks#Created=1669304319#Command=sh -c '/root/setup.sh ; service bind9 start ; /root/dyndns'#Image=bbaerthlein/docker-ddns-server:latest#ImageID=sha256:c16d9c8f136bb426652723165e3510d49bf01ed8886d9a27d8f4e885e3ce7c8d#SizeRootFs=200647059#SizeRw=734#CPU_usage=134674993588.000000#CPU_system_usage=29042541750000000.000000#Memory_used=101462016#Memory_limit=26267291648
/wg-easy#Names=/wg-easy#State=running#Status=Up 6 weeks#Created=1664568399#Command=docker-entrypoint.sh /usr/bin/dumb-init node server.js#Image=weejewel/wg-easy#ImageID=sha256:c90b263d8a383ac55905e4edbe1513ba87b1a479bcaa8bb471412137b9541751#SizeRootFs=141586777#CPU_usage=1094803796.000000#CPU_system_usage=29042549760000000.000000#Memory_used=65396736#Memory_limit=26267291648
/portainer#Names=/portainer#State=running#Status=Up 6 weeks#Created=1628164251#Command=/portainer#Image=portainer/portainer-ce#ImageID=sha256:dfac2df130448c2286f577276601d2d2da2f1aba5fbdfb729924eb5d24457c67#SizeRootFs=209674551#CPU_usage=823707359852.000000#CPU_system_usage=29042557780000000.000000#Memory_used=52162560#Memory_limit=26267291648
<<<docker_images:sep(35)>>>
 -- all the informations about the images --
Bastian-Kuhn commented 8 months ago

Hello,

can you add the Full Traceback so that I can see what goes wrong in the end?

Semetra22 commented 8 months ago

Hey @Bastian-Kuhn thanks for the blazing fast reply! Unfortunately, this is the complete traceback that I see in the check-mk interface.

But I have attached the complete crash report here:

http://assets.heko-technik.de/dl/Checkmk_Crash_monitoring_75f36b2c-7e4a-11ee-b09c-83a20f8124f9_2023-11-08_15-53-26.tar.gz

Bastian-Kuhn commented 8 months ago

Is fixed in the newest Version

Semetra22 commented 8 months ago

Many thanks for the quick help! The fix has solved the problem and I can now monitor all services.

Marked as solved