search

monitoringartist / dockbix-agent-xxl

:whale: Dockerized Zabbix agent with Docker metrics and host metrics support for CoreOS, RHEL, CentOS, Ubuntu, Debian, Fedora, Boot2docker, Photon OS, Amazon Linux, ...
https://hub.docker.com/r/monitoringartist/dockbix-agent-xxl-limited/
Other
182 stars 54 forks source link

Server can't get any info from agent #47

Closed Unyxos closed 5 years ago

Unyxos commented 5 years ago

Hi! We installed Dockbix-Agent-XXL (latest version) on our Kubernetes cluster today, according to the logs it's just running fine, and our zabbix server (which is not hosted on the cluster) can detect the agent.

However, when trying to get any information from the agent, we just receive this : "invalid item key format" (we even tried to get the agent.version value, which also returns invalid item key format).

Is it impossible to deploy Dockbix on Kubernetes yet? I also want to mention our server is using Zabbix 4, but according to Zabbix's doc, older agents are supposed to work..

Thank you for your reply!

jangaraj commented 5 years ago

Please increase debug level https://github.com/monitoringartist/dockbix-agent-xxl#support and provide pod logs.

Unyxos commented 5 years ago

Hi, I increased the debug level and that's what i'm getting, I noticed the undefined symbols errors but I'm not sure how to fix them as I'm using the provided image from Docker Hub. (I replaced all our hostnames/IP addresses with other values)

Dockbix Agent XXL v0.0.5b limited version
Docker image: monitoringartist/dockbix-agent-xxl-limited
Copyright (C) 2014-2017 Jan Garaj - www.monitoringartist.com
Freeware licence - Usage of this binary is restricted to official monitoringartist Docker images only.
Starting Zabbix Agent [dockbix-agent-xxl-6899cb7bb8-hhn5c]. Zabbix 3.4.2 Dockbix Agent XXL (2017-09-26) (revision 72885).
Press Ctrl+C to exit.
      9:20190220:074151.067 Starting Zabbix Agent [dockbix-agent-xxl-6899cb7bb8-hhn5c]. Zabbix 3.4.2 Dockbix Agent XXL (2017-09-26) (revision 72885).
     9:20190220:074151.067 **** Enabled features ****
     9:20190220:074151.067 IPv6 support:          YES
     9:20190220:074151.067 TLS support:           YES
     9:20190220:074151.067 **************************
     9:20190220:074151.067 using configuration file: /etc/zabbix/zabbix_agentd.conf
     9:20190220:074151.067 In zbx_load_modules()
     9:20190220:074151.067 loading module "/var/lib/zabbix/modules/zabbix_module_docker.so"
     9:20190220:074151.067 In zbx_module_api_version()
     9:20190220:074151.067 In zbx_module_init()
     9:20190220:074151.067 zabbix_module_docker v0.6.8, compilation time: Sep 26 2017 22:48:15
     9:20190220:074151.067 In zbx_docker_dir_detect()
     9:20190220:074151.067 Cannot detect used docker driver
     9:20190220:074151.067 In zbx_docker_api_detect()
     9:20190220:074151.067 In zbx_module_docker_socket_query()
     9:20190220:074151.067 Docker's socket query: GET /_ping HTTP/1.0
     9:20190220:074151.068 Docker's socket response: OK
     9:20190220:074151.068 Docker's socket works - extended docker metrics are available
     9:20190220:074151.068 In zbx_module_item_list()
     9:20190220:074151.068 In zbx_module_item_timeout()
     9:20190220:074151.068 cannot find "zbx_module_history_write_cbs()" function in module "zabbix_module_docker.so": /var/lib/zabbix/modules/zabbix_module_docker.so: undefined symbol: zbx_module_history_write_cbs
     9:20190220:074151.068 loading module "/var/lib/zabbix/modules/zabbix_module_stress.so"
     9:20190220:074151.068 cannot find "zbx_module_history_write_cbs()" function in module "zabbix_module_stress.so": /var/lib/zabbix/modules/zabbix_module_stress.so: undefined symbol: zbx_module_history_write_cbs
     9:20190220:074151.070 loaded modules: zabbix_module_docker.so, zabbix_module_stress.so, zabbix_module_systemd.so, zabbix_module_sockets.so
     9:20190220:074151.070 End of zbx_load_modules():SUCCEED
     9:20190220:074151.070 In init_collector_data()
     9:20190220:074151.070 In zbx_dshm_create() size:0
     9:20190220:074151.070 End of zbx_dshm_create():SUCCEED shmid:-1
     9:20190220:074151.070 End of init_collector_data()
     9:20190220:074151.070 agent #0 started [main process]
    15:20190220:074151.073 agent #4 started [listener #3]
    15:20190220:074151.073 In zbx_tls_init_child()
    12:20190220:074151.073 agent #1 started [collector]
    12:20190220:074151.073 In init_cpu_collector()
    12:20190220:074151.073 End of init_cpu_collector():SUCCEED
    12:20190220:074151.073 __zbx_zbx_setproctitle() title:'collector [processing data]'
    12:20190220:074151.073 In update_cpustats()
    12:20190220:074151.073 End of update_cpustats()
    12:20190220:074151.073 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
    14:20190220:074151.073 agent #3 started [listener #2]
    14:20190220:074151.073 In zbx_tls_init_child()
    13:20190220:074151.074 agent #2 started [listener #1]
    13:20190220:074151.074 In zbx_tls_init_child()
    15:20190220:074151.075 OpenSSL library (version OpenSSL 1.0.2g  1 Mar 2016) initialized
    13:20190220:074151.075 OpenSSL library (version OpenSSL 1.0.2g  1 Mar 2016) initialized
    13:20190220:074151.075 End of zbx_tls_init_child()
    15:20190220:074151.075 End of zbx_tls_init_child()
    13:20190220:074151.075 __zbx_zbx_setproctitle() title:'listener #1 [waiting for connection]'
    15:20190220:074151.075 __zbx_zbx_setproctitle() title:'listener #3 [waiting for connection]'
    16:20190220:074151.075 agent #5 started [active checks #1]
    16:20190220:074151.075 In zbx_tls_init_child()
    14:20190220:074151.076 OpenSSL library (version OpenSSL 1.0.2g  1 Mar 2016) initialized
    14:20190220:074151.076 End of zbx_tls_init_child()
    14:20190220:074151.076 __zbx_zbx_setproctitle() title:'listener #2 [waiting for connection]'
    16:20190220:074151.077 OpenSSL library (version OpenSSL 1.0.2g  1 Mar 2016) initialized
    16:20190220:074151.077 End of zbx_tls_init_child()
    16:20190220:074151.077 In init_active_metrics()
    16:20190220:074151.077 buffer: first allocation for 100 elements
    16:20190220:074151.077 End of init_active_metrics()
    16:20190220:074151.077 In send_buffer() host:'<Zabbix Server's IP>' port:10051 entries:0/100
    16:20190220:074151.077 End of send_buffer():SUCCEED
    16:20190220:074151.077 __zbx_zbx_setproctitle() title:'active checks #1 [getting list of active checks]'
    16:20190220:074151.077 In refresh_active_checks() host:'<Zabbix Server's IP>' port:10051
    16:20190220:074151.078 sending [{"request":"active checks","host":"dockbix-agent-xxl-6899cb7bb8-hhn5c","host_metadata":"dockbix-agent-xxl"}]
    16:20190220:074151.078 before read
    16:20190220:074151.081 got [{"response":"failed","info":"host [dockbix-agent-xxl-6899cb7bb8-hhn5c] not found"}]
    16:20190220:074151.081 In parse_list_of_checks()
    16:20190220:074151.081 no active checks on server [<Zabbix Server's IP>:10051]: host [dockbix-agent-xxl-6899cb7bb8-hhn5c] not found
    16:20190220:074151.081 End of parse_list_of_checks():FAIL
    16:20190220:074151.081 End of refresh_active_checks():SUCCEED
    16:20190220:074151.081 __zbx_zbx_setproctitle() title:'active checks #1 [processing active checks]'
    16:20190220:074151.081 In process_active_checks() server:'<Zabbix Server's IP>' port:10051
    16:20190220:074151.081 End of process_active_checks()
    16:20190220:074151.081 In get_min_nextcheck()
    16:20190220:074151.081 End of get_min_nextcheck():-1
    16:20190220:074151.081 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
    12:20190220:074152.073 __zbx_zbx_setproctitle() title:'collector [processing data]'
    12:20190220:074152.073 In update_cpustats()
    12:20190220:074152.074 End of update_cpustats()
    12:20190220:074152.074 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
    16:20190220:074152.081 In send_buffer() host:'<Zabbix Server's IP>' port:10051 entries:0/100
    16:20190220:074152.081 End of send_buffer():SUCCEED

After that it just continues with the active checks lines. The pod has everything needed (volumes, privileged...) according to Dockbix's documentation. (I changed the hostname on zabbix's side after deploying the pod so I have the correct hostname now)

On the Zabbix Server's host : 

[centos@zabbixserver ~]$ sudo zabbix_get -s <Agent Address> -k agent.version
ZBX_NOTSUPPORTED: Invalid item key format.

And the agent appears as available on our Zabbix Server's dashboard

jangaraj commented 5 years ago

Unfortunately, you need dev cgroup-parent branch of zabbix docker module https://github.com/monitoringartist/zabbix-docker-monitoring/tree/cgroup-parent for K8S container monitoring. This module version is not available in the Docker image dockbix-agent-xxl.