search

function61 / promswarmconnect

Bridges Docker Swarm services to Prometheus without any changes to Prometheus
https://function61.com/
Apache License 2.0
24 stars 6 forks source link

Expose swarm hostname #4

Closed ainsey11 closed 5 years ago

ainsey11 commented 5 years ago

Love this so far, however is it possible to differentiate which node the metrics instance is running on? a prime example of this would be cadvisor,

say I have a three node cluster, I end up with three discovered cadvisor containers, something a little like image

Is there a way I can tell prometheus which node each of these lives on? the only way I can differentate between the three is the vm_alias field which returns the containers IP address, this would be very useful with things like grafana etc,

Thanks! Ainsey

joonas-fi commented 5 years ago

Glad you found this useful!

If I understood correctly, yes, I'm also doing that because I have node-specific metrics I want to observe in production.

If you have a per Swarm-node service and you want to have it show up in Prometheus by the server hostname as it's instance value, you can override it like this: https://github.com/function61/promswarmconnect/blob/7b450c4767c12f001b0d0318bf4bff68fffbfaa7/cmd/promswarmconnect/metricsendpointparser_test.go#L68

Basically, when creating Swarm service add an ENV var: $ docker service create -e METRICS_ENDPOINT=/metrics,instance=_HOSTNAME_

The _HOSTNAME_ is a magical placeholder which promswarmconnect will replace at discovery time.

ainsey11 commented 5 years ago

Thanks! That worked perfectly, it now returns "vm_uuid": "nc-docker-1" etc,

joonas-fi commented 5 years ago

Great!

I need to work on the docs to explain these better. This override was linked to from the README but it's not clear enough, and it even refers to a deprecated way of doing it (different ENV variable, which I would like to remove)

Have a nice weekend!

On Fri, Mar 1, 2019, 15:56 Ainsey11 notifications@github.com wrote:

Thanks! That worked perfectly, it now returns "vm_uuid": "nc-docker-1" etc,

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/function61/promswarmconnect/issues/4#issuecomment-468672625, or mute the thread https://github.com/notifications/unsubscribe-auth/AAmdh3_pT40Sn2G5f6b-VELuHaS-SIQpks5vSTGXgaJpZM4bYKvY .

ainsey11 commented 5 years ago

No worries,

I was in the progress of tweaking the readme to make it a little clearer, hopefully that helps!