search

monitoringartist / dockbix-agent-xxl

:whale: Dockerized Zabbix agent with Docker metrics and host metrics support for CoreOS, RHEL, CentOS, Ubuntu, Debian, Fedora, Boot2docker, Photon OS, Amazon Linux, ...
https://hub.docker.com/r/monitoringartist/dockbix-agent-xxl-limited/
Other
182 stars 54 forks source link

Error response from daemon: Unable to remove filesystem for ab68b000adfca007 ... #11

Closed ksprider closed 8 years ago

ksprider commented 8 years ago

Hello.

docker-compose.yml

zabbix-agent-xxl:
    image: monitoringartist/zabbix-agent-xxl-limited:latest
    hostname: agent1
    ports:
        - "10050:10050"
    volumes:
        - /:/rootfs
        - /var/run:/var/run
    environment:
        - ZA_Server=192.168.2.176
        - ZA_ServerActive=192.168.2.176
docker-compose up -d

output:

[root@localhost zabbix-agent]# docker-compose up -d
Creating zabbixagent_zabbix-agent-xxl_1
Error response from daemon: Unable to remove filesystem for ab68b000adfca007cfa18d2f4c12c31c60818ace86c27c3a61afd53d6d05e073: remove /var/lib/docker/containers/ab68b000adfca007cfa18d2f4c12c31c60818ace86c27c3a61afd53d6d05e073/shm: device or resource busy
[root@localhost zabbix-agent]#

OS:

LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.2.1511 (Core) 
Release:        7.2.1511
Codename:       Core
[root@localhost zabbix-agent]# docker logs b5792366ef21
Zocker XXL v0.9.3b public limited version
Copyright (C) 2014-2016 Jan Garaj - www.monitoringartist.com 
Freeware licence - Usage of this binary is restricted to oficial monitoringartist Docker images only.
Starting Zabbix Agent [agent1]. Zabbix 3.0.3 XXL (2016-06-16) (revision {ZABBIX_REVISION}).
Press Ctrl+C to exit.

     8:20160619:181500.031 Starting Zabbix Agent [agent1]. Zabbix 3.0.3 XXL (2016-06-16) (revision {ZABBIX_REVISION}).
     8:20160619:181500.032 **** Enabled features ****
     8:20160619:181500.032 IPv6 support:          YES
     8:20160619:181500.032 TLS support:           YES
     8:20160619:181500.032 **************************
     8:20160619:181500.032 using configuration file: /etc/zabbix/zabbix_agentd.conf
     8:20160619:181500.040 loaded modules: zabbix_module_docker.so, zabbix_module_stress.so
     8:20160619:181500.041 agent #0 started [main process]
    10:20160619:181500.042 agent #1 started [collector]
    11:20160619:181500.043 agent #2 started [listener #1]
    12:20160619:181500.043 agent #3 started [listener #2]
    13:20160619:181500.044 agent #4 started [listener #3]
    14:20160619:181500.044 agent #5 started [active checks #1]
    14:20160619:181503.058 active check configuration update from [192.168.2.176:10051] started to fail (ZBX_TCP_READ() timed out)
[root@localhost zabbix-agent]# docker -v
Docker version 1.11.2, build b9f10c9
[root@localhost zabbix-agent]# telnet 192.168.2.176 10051
Trying 192.168.2.176...
Connected to 192.168.2.176.
Escape character is '^]'.
Connection closed by foreign host.
ksprider commented 8 years ago
jangaraj commented 8 years ago

Probably some zabbix container was already running. Be sure that all zabbix containers are killed and removed, before starting new one.

ksprider commented 8 years ago 
[root@localhost zabbix-agent]# docker ps -a
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS              PORTS                                                              NAMES
a17b8dd46a8d        monitoringartist/zabbix-3.0-xxl:3.0.3   "/config/bootstrap.sh"   2 hours ago         Up 2 hours          0.0.0.0:80->80/tcp, 0.0.0.0:10051->10051/tcp, 162/udp, 10052/tcp   zabbix_zabbix_1
efc59de25dba        mysql                                   "docker-entrypoint.sh"   11 hours ago        Up 11 hours         0.0.0.0:3306->3306/tcp                                             mysql_mysql_1
[root@localhost zabbix-agent]# docker-compose up -d
Creating zabbixagent_zabbix-agent-xxl_1
Error response from daemon: Unable to remove filesystem for 669033a03ee3b8102a781d57da5cb36cc0e30dfaa8fcbe28f20ddadc8b6c9246: remove /var/lib/docker/containers/669033a03ee3b8102a781d57da5cb36cc0e30dfaa8fcbe28f20ddadc8b6c9246/shm: device or resource busy
[root@localhost zabbix-agent]#

zabbix-3.0-xxl is killed and removed.

[root@localhost zabbix-agent]# docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
efc59de25dba        mysql               "docker-entrypoint.sh"   11 hours ago        Up 11 hours         0.0.0.0:3306->3306/tcp   mysql_mysql_1
[root@localhost zabbix-agent]# docker-compose up -d
Creating zabbixagent_zabbix-agent-xxl_1
Error response from daemon: Unable to remove filesystem for 2cac7882b41dfb3b23490b1eb059d136591887197e00d003538ee800fda6991b: remove /var/lib/docker/containers/2cac7882b41dfb3b23490b1eb059d136591887197e00d003538ee800fda6991b/shm: device or resource busy
[root@localhost zabbix-agent]#
jangaraj commented 8 years ago

Unfortunately, I'm still not able to replicate your problem. My test: OS: CentOS Linux release 7.2.1511 (Core) Docker: Docker version 1.11.2, build b9f10c9 Docker compose: docker-compose version 1.6.2, build 4d72027 Zabbix agent version: Zabbix 3.0.3 XXL (2016-06-16) your docker-compose.yaml:

[root@centos-512mb-lon1-01 del]# docker-compose up -d
Creating del_zabbix-agent-xxl_1
[root@centos-512mb-lon1-01 del]# docker ps
CONTAINER ID        IMAGE                                              COMMAND             CREATED             STATUS              PORTS                      NAMES
2df566176ab7        monitoringartist/zabbix-agent-xxl-limited:latest   "/zocker-xxl"       5 seconds ago       Up 5 seconds        0.0.0.0:10050->10050/tcp   del_zabbix-agent-xxl_1
[root@centos-512mb-lon1-01 del]# docker logs 2df566176ab7
Zocker XXL v0.9.3b public limited version
Copyright (C) 2014-2016 Jan Garaj - www.monitoringartist.com
Freeware licence - Usage of this binary is restricted to official monitoringartist Docker images only.
Starting Zabbix Agent [agent1]. Zabbix 3.0.3 XXL (2016-06-16) (revision {ZABBIX_REVISION}).
Press Ctrl+C to exit.

     8:20160622:193904.630 Starting Zabbix Agent [agent1]. Zabbix 3.0.3 XXL (2016-06-16) (revision {ZABBIX_REVISION}).
     8:20160622:193904.630 **** Enabled features ****
     8:20160622:193904.630 IPv6 support:          YES
     8:20160622:193904.630 TLS support:           YES
     8:20160622:193904.630 **************************
     8:20160622:193904.630 using configuration file: /etc/zabbix/zabbix_agentd.conf
     8:20160622:193904.632 loaded modules: zabbix_module_docker.so, zabbix_module_stress.so
     8:20160622:193904.632 agent #0 started [main process]
    10:20160622:193904.633 agent #1 started [collector]
    11:20160622:193904.633 agent #2 started [listener #1]
    12:20160622:193904.635 agent #3 started [listener #2]
    13:20160622:193904.637 agent #4 started [listener #3]
    14:20160622:193904.639 agent #5 started [active checks #1]
    14:20160622:193907.641 active check configuration update from [192.168.2.176:10051] started to fail (cannot connect to [[192.168.2.176]:10051]: [4] Interrupted system call)
[root@centos-512mb-lon1-01 del]# docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 1.11.2
Storage Driver: devicemapper
 Pool Name: docker-253:1-261720-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 199.5 MB
 Data Space Total: 107.4 GB
 Data Space Available: 19.64 GB
 Metadata Space Used: 696.3 kB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.147 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2015-12-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null host bridge
Kernel Version: 3.10.0-327.18.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 489.1 MiB
Name: centos-512mb-lon1-01
ID: ETXC:XTYQ:T56K:LFKD:INZQ:SBED:4CLA:2F5Y:3V3U:FSMG:CAHV:EQUQ
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

[192.168.2.176:10051] started to fail (ZBX_TCP_READ() timed out) -> has your container network connectivity to the internet and to 192.168.2.176:10051?

jangaraj commented 8 years ago

Closing - no response from the user.

yujinov commented 7 years ago

I has same issue and fixed it the next way: Change

 volumes:
        - /:/rootfs

to:

 volumes:
        #Mount some unused folder from host filesystem for monitoring disk space.
        - /usr/local/etc:/rootfs:ro
        #Restrict access to host /var folder from privileged container
        - /var
jangaraj commented 7 years ago

/usr/local/etc:/rootfs:ro - that's definitely wrong - there is required Docker socket, which needs RW access.