search

sensu-plugins / sensu-plugins-docker

This plugin provides native Docker instrumentation for monitoring and metrics collection, including: container status, container number, and container metrics via `docker ps`.
http://sensu-plugins.io
MIT License
35 stars 57 forks source link

Does this plugin support docker Swarm? #54

Closed sherabi closed 7 years ago

sherabi commented 7 years ago

Does this work with Docker Swarm?

majormoses commented 7 years ago

I do not see anything swarm specific are you talking about swarm specific metrics? It will work with any docker engine new enough regardless of the scheduler but if you are looking for scheduler metrics it does not currently.

sherabi commented 7 years ago

I am trying to run on a host that is running swarm manually and get this....

./metrics-docker-stats.rb -H 10.127.84.73:2377
Check failed to run: wrong status line: "\x15\x03\x01\x00\x02\x02", ["/opt/sensu/embedded/lib/ruby/2.3.0/net/http/response.rb:42:in `read_status_line'", "/opt/sensu/embedded/lib/ruby/2.3.0/net/http/response.rb:29:in `read_new'", "/opt/sensu/embedded/lib/ruby/2.3.0/net/http.rb:1437:in `block in transport_request'", "/opt/sensu/embedded/lib/ruby/2.3.0/net/http.rb:1434:in `catch'", "/opt/sensu/embedded/lib/ruby/2.3.0/net/http.rb:1434:in `transport_request'", "/opt/sensu/embedded/lib/ruby/2.3.0/net/http.rb:1407:in `request'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-docker-1.3.1/bin/metrics-docker-stats.rb:146:in `block in docker_api'", "/opt/sensu/embedded/lib/ruby/2.3.0/net/http.rb:853:in `start'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-docker-1.3.1/bin/metrics-docker-stats.rb:145:in `docker_api'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-docker-1.3.1/bin/metrics-docker-stats.rb:156:in `list_containers'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugins-docker-1.3.1/bin/metrics-docker-stats.rb:108:in `run'", "/opt/sensu/embedded/lib/ruby/gems/2.3.0/gems/sensu-plugin-1.4.2/lib/sensu-plugin/cli.rb:58:in `block in <class:CLI>'"]
majormoses commented 7 years ago

@sherabi I will try to take a look this weekend, can you give me the output of docker version?

sherabi commented 7 years ago 
docker version
Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Mon Mar 27 17:05:44 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Mon Mar 27 17:05:44 2017
 OS/Arch:      linux/amd64
 Experimental: false
majormoses commented 7 years ago

sorry I did not have any time this weekend to look at this, I should have some time this week. Just checking have you enabled remote docker api access per: https://docs.docker.com/engine/installation/linux/linux-postinstall/#allow-access-to-the-remote-api-through-a-firewall

sherabi commented 7 years ago

@majormoses yes remote docker api is enabled, I have temporarily turned off firewall anyways. So I am certain firewalls aren't the issue.

majormoses commented 7 years ago

Hmm I was unable to replicate using a unix socket (no reccomending to give access to the docker socket as that is dangerous):

$ ./bin/metrics-docker-stats.rb -H /var/run/docker.sock -p unix
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.preread 2017-07-31T23:53:17.287379874Z 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.pids_stats.current 1 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.num_procs 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.cpu_stats.cpu_usage.total_usage 18225944 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.cpu_stats.cpu_usage.usage_in_kernelmode 10000000 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.cpu_stats.cpu_usage.usage_in_usermode 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.cpu_stats.system_cpu_usage 222321230000000 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.cpu_stats.online_cpus 8 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.cpu_stats.throttling_data.periods 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.cpu_stats.throttling_data.throttled_periods 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.cpu_stats.throttling_data.throttled_time 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.precpu_stats.cpu_usage.total_usage 18225944 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.precpu_stats.cpu_usage.usage_in_kernelmode 10000000 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.precpu_stats.cpu_usage.usage_in_usermode 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.precpu_stats.system_cpu_usage 222313320000000 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.precpu_stats.online_cpus 8 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.precpu_stats.throttling_data.periods 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.precpu_stats.throttling_data.throttled_periods 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.precpu_stats.throttling_data.throttled_time 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.usage 5234688 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.max_usage 5492736 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.active_anon 507904 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.active_file 3661824 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.cache 4751360 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.dirty 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.hierarchical_memory_limit 9223372036854771712 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.inactive_anon 28672 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.inactive_file 1036288 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.mapped_file 2727936 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.pgfault 905 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.pgmajfault 34 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.pgpgin 1659 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.pgpgout 381 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.rss 483328 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.rss_huge 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_active_anon 507904 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_active_file 3661824 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_cache 4751360 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_dirty 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_inactive_anon 28672 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_inactive_file 1036288 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_mapped_file 2727936 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_pgfault 905 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_pgmajfault 34 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_pgpgin 1659 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_pgpgout 381 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_rss 483328 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_rss_huge 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_unevictable 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.total_writeback 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.unevictable 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.stats.writeback 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.memory_stats.limit 33692499968 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.name /vibrant_minsky 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.id 61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.networks.eth0.rx_bytes 5551 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.networks.eth0.rx_packets 39 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.networks.eth0.rx_errors 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.networks.eth0.rx_dropped 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.networks.eth0.tx_bytes 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.networks.eth0.tx_packets 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.networks.eth0.tx_errors 0 1501545196
babrams-Serval-WS.docker.61fed32ca107cc4c7080d08ee9e8304d9b90f414b44fee3009069ba0f80c975e.networks.eth0.tx_dropped 0 1501545196
majormoses commented 7 years ago

Before I enable the docker api locally for me to test can you please verify that you have enabled the docker remote api by pasting the following output (sadly I am assuming systemd):

$ sudo grep ExecStart /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd://`

It is possible you might have configured a host in /etc/default/docker so include any other configs, env vars, etc. that you might have configured.

If you see output like mine you have not enabled docker remote api. and will want to modify it to something like this: -H tcp://0.0.0.0:4243 although probably better to at least restrict its listening ip to something private like the loopback address. If you do this please enable your firewall (iptables, ufw, etc) and have very strict rules. I would discourage anyone from allowing it from everywhere as Having a docker api available over the network is very scary and in the wrong hands terrible things can be done which is why the default docker setup does not listen on any ips and only works with the local unix socket. Here is an article talking a little about how to setup docker remote api: https://www.ivankrizsan.se/2016/05/18/enabling-docker-remote-api-on-ubuntu-16-04/

sherabi commented 7 years ago

@majormoses A simple restart of the docker daemon got the plugin working for me. I didn't change anything related to the docker configs and /etc/default/docker is non-existent.

majormoses commented 7 years ago

glad it's working now.