Ability to set a tag on Consul's own service registration

[far-blue]

It would be useful if consul was able to add tags to its own service registration. This could be to allow referring to particular instances by DNS or it could be for external uses such as Fabio proxy routing.

[slackpad]

Hi @far-blue - what uses were you thinking of? We probably should just have Consul tag itself automatically for most things (leader, follower seems like a good one for servers), any other use cases you had in mind?

[far-blue]

leader / follower is a good example for an automated tag but tagging is also helpful just as labels for managing resources. Some third party apps such as Fabio and anything built using consul-template to react to service data also tend to use tags to identify or exclude services.

[slackpad]

Are you looking to add tags related to Consul's service itself, though, or are you more looking for tags that represent the node as a whole?

[far-blue]

That's a good question. I guess I was thinking in terms of Consul's Rest API as a service - much like Nomad and Vault present their API interface(s) as services and that is still my interest.

However, I guess it is also fair to say that nodes themselves could have tags rather than the consul service per-se. I guess this would be closer to Nomad's 'meta' info for agents. Grouping of nodes based on a tag would allow dns queries for nodes similar to services.

[bfgoodrich]

As far as tags for nodes - https://github.com/hashicorp/consul/issues/154

[bacoboy]

A good use case for this is when using traefik's consul_catalog.

Consul's self registration gets picked up as a real service on port 8300. The way you tell traefik to ignore this is by setting a tag traefik.enable=false on the service (like traefik does to its own services):

% curl -s http://127.0.0.1:8500/v1/catalog/services | jq
{
  "consul": [],
  "traefik-dashboard": [
    "traefik.enable=false"
  ],
  "traefik-http": [
    "traefik.enable=false"
  ]
}

I stumbled across this issue looking for how to set a tag on consul's self-registration 😉

[sev3ryn]

I'm using tags to point Prometheus what services to scrape for metrics and what metric_path to use - this would be useful to be able to scrape from Consul's /v1/agent/metrics?format=prometheus endpoint

[jinnatar]

Any updates to this one? While I can manage any service in Consul with tags to configure it for Traefik, ironically I cannot manage the Consul UI itself since I cannot add tags to it that would then be consumed by the Traefik consul_catalog. I can configure it as a static Traefik routing but then lose health check based load balancing.

[blmhemu]

Hello @Artanicus ! May I ask, if you can share your static config. I am looking for something similar and came across this...

[itorres]

@blmhemu something like this will expose the local consul agent (and in our case also nomad):

http:                                    
  routers:                               
    consul:                              
      rule: "Host(`consul.example.net`)"
      service: "consul"                  
    nomad:                               
      rule: "Host(`nomad.example.net`)" 
      service: "nomad"                   
  services:                              
    consul:                              
      loadBalancer:                      
        servers:                         
          - url: "http://localhost:8500" 
    nomad:                               
      loadBalancer:                      
        servers:                         
          - url: "http://localhost:4646" 

[blmhemu]

@itorres Thanks for the reply ! In the meanwhile, while searching on web, I think I may have found another solution (thanks to Michael Aldrige) using consul's service discovery. http://localhost:4646 can also be written as http://nomad.service.consul:4646.

[dnephin]

Cross posting a comment I left on a duplicate issue about how this might be implemented.

We discussed this a bit. It us my understanding that this could actually be challenging to implement correctly because of how Consul registers itself as a service. When a Consul node starts it does not itself perform an RPC call to register itself in the Catalog. Instead, what happens is that the gossip "node join" event causes the leader to register the node in the Catalog. So to get these tags into the Catalog the tags would have to be added to the serf tags. This is a problem because serf message have a fixed maximum size, so someone trying to add many tags could exceed the maximum size and cause the node join to fail.

It's difficult to determine the maximum size for tags because we don't know at the time of config validation the size of the other parts of the message.

Two options for addressing this problem would be:

We could introduce some very strict limits for the tags (both number of tags, and maximum size for the key and value) We could find a different way of registering nodes in the Catalog. I believe the anti-entropy sync also updates the node registration. Maybe that could be used to update the tags?

If we do it as part of the anti-entropy sync, we could add the tags here: https://github.com/hashicorp/consul/blob/v1.10.3/agent/local/state.go#L1360-L1370

But @rboyer pointed out that for this to work the agent token would need additional privileges to update the consul service .

[chrisvanmeer]

Any new thoughts on this? My use case would for using this with Traefik, but I don't want to expose all my services by default so I should be able to manually add tags to the own consul service.

For now I will have to stick to a file provider and manually configure the Consul servers in there.