Closed thomasf closed 5 years ago
@tcolgate could you comment on the user facing behavior stuff I wrote about in this issue description.. I would like to have most of it decided before I'm committing my time towards the underlying solution.
You can ignore any actual code for now, I'll inform you when it's ready for a discussion.
Sorry I dropped the ball on this, I'll give this a review
(QUESTION) should it be allowed to configure the same service multiple times?
It's questionable if multi identical key sections in yaml is valid. I've seen app drop earlier sections, and I've also got a recollection that the latest go-yaml considers this an actual error. I wouldn't bother.
myservice2:
method: http
http:
port: 8100 # myservice2:1 or myservice2:8100
The ambiguity of have name:ordinal and name:port makes me a little uneasy.whilst overlap is incredibly unlikely, it could be confusing to users.
re: mixing tls and non-tls socks, I'm not too fussed. you could cofigure two seperate listeners I suppose, but I think if someone is already enforcing socks and tls, they are probably fully invested.
regarding -socks.prefix cool.host1
, I think it may be better to just support general addition of a label to the returned metrics. Each exporter could then be configured to add its own "proxy_instance" label then
@thomasf do you have an actual production use case for the port ranges?
Yes, python application clusters where each process has a separate prometheus exporter instance running on it;s own thread. Multiple small customers with smaller and larger VPS'es around the world with different "cloud" providers and one prometheus instance to collect it all.
I'll update the spec tomorrow based on your comments.
thanks, generally in favour. It does feel like maybe it's got a few extra features that aren't specific to socks implementation.
While this has worked in practice for a very long time in production i'm closing this issue becase I don't have any reason to continue working on it.. We have changed to just using a wildcard domain for each host so that each prometheus exporter gets it'd own https subdomain instead.
if anyone is interested we monitor like this now
- job_name: "minio-foobar-staging"
scheme: https
metrics_path: '/minio/prometheus/metrics'
static_configs:
- targets: ['s.foobar.staging.example.com']
labels:
context: 'foobar.staging'
- job_name: "netdata-foobar-staging"
basic_auth:
username: prometheus
password: 'somepassword'
scheme: https
metrics_path: '/api/v1/allmetrics'
params:
format: ['prometheus']
static_configs:
- targets: ['netdata.monitor.foobar.staging.example.com']
labels:
context: 'foobar.staging'
- job_name: "traefik-foobar-staging"
basic_auth:
username: prometheus
password: 'somepassword'
scheme: https
static_configs:
- targets: ['traefik.monitor.foobar.staging.example.com']
labels:
context: 'foobar.staging'
- job_name: "django-foobar-staging"
basic_auth:
username: prometheus
password: 'somepassword'
scheme: https
static_configs:
- targets: ['app1.monitor.foobar.staging.example.com', 'app2.monitor.foobar.staging.example.com', 'crawl1.monitor.foobar.staging.example.com','notify1.monitor.foobar.staging.example.com']
labels:
context: 'foobar.staging'
Work in progress for #9
basic configuration
I'm working towards the configuration style below. The
ports:
field is based on code for a docker-compose.ymlports:"
parser from some docker package..easy/safe unique scrape host names
The config below would create the same scrape target instance names
myservice:1
andmyservice:2
for multiple actual hosts which isn't great and can lead to accidental mix ups in Prometheus queries.While it is probably possible to avoid instance name collisions using relabeling inside Prometheus I think it's best to be able to ensure instance name uniqueness at the node level.
To solve this issue an optional
-socks.prefix
flag is introduced toexporter_exporter
which enforces a FQN prefix to the virtual host names.If an
exporter_exporter
instance is started with-socks.prefix cool.host1
the scrape target would then look like this:To lessen the requirements of manual node configuration An additional boolean flag
-socks.auto-prefix
will try to set the prefix usingos.Hostname()
returns unless it returns an empty string or error in which case start up will be aborted by an fatal exit.command line options
right now it looks like this (the descriptions will be revised later):
(QUESTION) can not use socks method + TLS + non-TLS at the same time
Because
-socks.tls
and-socks.listen-address
are combined the socks method can not be enabled for both tls and non tls access simultaneously. I don't see it useful to have both working at the same time but we have to change some things if it should be supported.(QUESTION) should it be allowed to configure the same service multiple times?
I'm a bit skeptical of this feature since it complicates the rules for what a valid configuration can be while not adding strong practical benefits.
This question can be deferred since it can be added later if needed
For the directory based config
myservice.app1.yml
andmyservice.app2.yml
would work for allowing multiple instances.(QUESTION) exposing multi port services for non socks methods
This question can be deferred since it can be added later if needed
I'm not sure if if is meaningful to implement the multiple ports per service scheme for the regular http and https serving options via URL parameters. The corresponding prometheus.yml would become very complicated. Could be useful if one wants to use a web browser to easily view the exporters but that isn't easy when using client certificates login anyway so maybe not a big win?