Support kubernetes-valid characters for meta keys

[lkysow]

Feature Description

Currently the only allowed meta characters are [a-z0-9] and -, _ (see https://www.consul.io/docs/agent/options#_node_meta for full rules).

1. A maximum of 64 key/value pairs can be registered per node.
2. Metadata keys must be between 1 and 128 characters (inclusive) in length
3. Metadata keys must contain only alphanumeric, -, and _ characters.
4. Metadata keys must not begin with the consul- prefix; that is reserved for internal use by Consul.
5. Metadata values must be between 0 and 512 (inclusive) characters in length.
6. Metadata values for keys beginning with rfc1035- are encoded verbatim in DNS TXT requests, otherwise the metadata kv-pair is encoded according RFC1464.

For consul-k8s we'd like to set meta keys like consul.hashicorp.com/key but that's not currently supported. There is a reserved key-space with consul-, but only Consul can set those keys, not outside tooling.

I suggest we follow the k8s standards: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set to allow for potential mapping back-and-forth in the future (although due to how our current rules are looser in some cases this might still be hard).

Annotations are key/value pairs. Valid annotation keys have two segments: an optional prefix and name, separated by a slash (/). The name segment is required and must be 63 characters or less, beginning and ending with an alphanumeric character ([a-z0-9A-Z]) with dashes (-), underscores (_), dots (.), and alphanumerics between. The prefix is optional. If specified, the prefix must be a DNS subdomain: a series of DNS labels separated by dots (.), not longer than 253 characters in total, followed by a slash (/).

The k8s standards are looser in some cases but tighter in others in that they require the end of the key be an alphanumeric character, not a - or _ and the name portion can only be 64 chars. We could perhaps loosen our requirements to allow for the domain-style prefixes (consul.hashicorp.com/test) without restricting them.

Use Case(s)

This would allow us to have hostname-based scoping of meta keys similar to Kubernetes annotations.

Perhaps this would also be useful for tags.

[jkirschner-hashicorp]

@lkysow: for meta keys, why make the validation rules be "DNS-valid characters and /"? If I understand correctly from RFC 1464, Section 2 (Format of TXT record), "any printable ASCII character is permitted for the attribute name". (Note: there's some nuance documented there about handling characters like \, ", and =)

[lkysow]

Great question! I was thinking it would be good to conform to the Kubernetes rules in case we wanted to map things back and forth at some point. https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set has those rules which are actually more strict than I'm proposing here.

[jkirschner-hashicorp]

I haven't looked through the code to see what characters we currently support for meta keys. Have you? If so, it might be good to document here for future readers.

I'm interested in understanding whether this would be a loosening or a tightening of the current validation behavior. And assuming it's a loosening, I like the suggestion to loosen to k8s annotation standards (in case we want to maps things back and forth) rather than going all the way to RFC 1464 (easier to loosen than tighten).

[lkysow]

@jkirschner-hashicorp for sure, I've updated the description. The current rules are here: https://www.consul.io/docs/agent/options#_node_meta. This would be a loosening in some cases but a tightening in others.

[lkysow]

I just noticed that for our config entries we actually do support all characters in the meta k/v