python-diamond / Diamond

Diamond is a python daemon that collects system metrics and publishes them to Graphite (and others). It is capable of collecting cpu, memory, network, i/o, load and disk metrics. Additionally, it features an API for implementing custom collectors for gathering metrics from almost any source.
http://diamond.readthedocs.org/
MIT License
1.74k stars 602 forks source link

Multi-server support for NginxCollector #745

Closed pzystorm closed 2 years ago

pzystorm commented 4 years ago

Hi, the NginxCollector has no multi-server support. I have a very small patch to make this possible. I don't wanted to change too much code, so I leave the main code in collect() untouched and just build a small wrap around:

--- collectors/nginx/nginx.bkup 2017-10-24 11:06:40.000000000 +0200
+++ collectors/nginx/nginx.py   2020-03-25 13:30:31.897773062 +0100
@@ -56,6 +56,47 @@
         return default_config

     def collect(self):
+
+        if isinstance(self.config['req_host_header'], list) and isinstance(self.config['req_host'], list) and \
+           isinstance(self.config['req_port'], list) and isinstance(self.config['req_path'], list) and \
+           isinstance(self.config['req_ssl'], list) and isinstance(self.config['req_path'], list):
+       
+            list_req_host_header = list(self.config['req_host_header'])
+            list_req_host        = list(self.config['req_host'])
+            list_req_port        = list(self.config['req_port'])
+            list_req_path        = list(self.config['req_path'])
+            list_req_ssl         = list(self.config['req_ssl'])
+            list_path            = list(self.config['path'])
+            shouldLength = len(list_req_host_header)
+
+            if len(list_req_host_header) != shouldLength or len(list_req_host) != shouldLength or \
+               len(list_req_port) != shouldLength or len(list_req_path) != shouldLength or \
+               len(list_req_ssl) != shouldLength or len(list_path) != shouldLength:
+
+                self.log.error("Unable to parse array with length: %i" % shouldLength)
+                return
+
+            for x in range(0, shouldLength):
+                self.config['req_host_header'] = list_req_host_header[x]
+                self.config['req_host']        = list_req_host[x]
+                self.config['req_port']        = list_req_port[x]
+                self.config['req_path']        = list_req_path[x]
+                self.config['req_ssl']         = list_req_ssl[x]
+                self.config['path']            = list_path[x]
+                self.collect2()
+
+            self.config['req_host_header'] = list_req_host_header
+            self.config['req_host']        = list_req_host
+            self.config['req_port']        = list_req_port
+            self.config['req_path']        = list_req_path
+            self.config['req_ssl']         = list_req_ssl
+            self.config['path']            = list_path
+
+        else:
+            self.collect2()
+
+    def collect2(self):
+
         # Determine what HTTP scheme to use based on SSL usage or not
         if str(self.config['req_ssl']).lower() == 'true':
             scheme = 'https'

The config file can then look like this:

[collectors]
[[NginxCollector]]
enabled = True
req_host_header=localhost, localhost
req_host=localhost, localhost
req_port=10000, 11000
req_path=/server-status, /server-status
req_ssl=false, false
path=proxy, portal

The output to graphite is then something like this:

machines.myservername.proxy.active_connections 2 1585141471
machines.myservername.proxy.conn_accepted 0 1585141471
machines.myservername.proxy.conn_handled 0 1585141471
machines.myservername.proxy.req_handled 0 1585141471
machines.myservername.proxy.req_per_conn 20 1585141471
machines.myservername.proxy.act_reads 0 1585141471
machines.myservername.proxy.act_writes 1 1585141471
machines.myservername.proxy.act_waits 1 1585141471
machines.myservername.portal.active_connections 2 1585141471
machines.myservername.portal.conn_accepted 0 1585141471
machines.myservername.portal.conn_handled 0 1585141471
machines.myservername.portal.req_handled 1 1585141471
machines.myservername.portal.req_per_conn 1 1585141471
machines.myservername.portal.act_reads 0 1585141471
machines.myservername.portal.act_writes 1 1585141471
machines.myservername.portal.act_waits 1 1585141471
shortdudey123 commented 2 years ago

The comments on #675 were multiple instances of a collector.