Support prepared queries

[bensojona]

I don't believe this is currently possible as the feature was just released in Consul 0.6 and introduced a new endpoint (/v1/query). But, it would be awesome if you could take advantage of prepared queries to do datacenter failover for service discovery in consul-template.

[slackpad]

We've been thinking a bit about what this integration would look like (does CT make the query for you and then tie it to a session for cleanup)? If you avoid trying to define a query via CT, which is probably a good way to go, then you could definitely execute a query and iterate over the results, which would be useful. For now you'd have to use DNS which is pretty limiting.

[bensojona]

@slackpad makes sense!

[slackpad]

Noting the tricky part is that the execute endpoint doesn't support blocking semantics so we'd have to figure that out, either adding blocking which would be very complex given the possible behaviors of failing over, etc. or adding some kind of polling interval on the CT side. Need to think about the better UX there.

[jippi]

For my personal case, a per-entry (maybe?) interval for refreshing

e.g. {{ with service mysql "near=_agent" "refresh=60s" }}

[jippi]

or maybe poll a unprepared version (e.g. a service), and when that changes, refresh the prepared version

{{ with service mysql | prepared mysql_locality }} would refresh the prepared every time the other service trigger a change?

[msvechla]

Is there currently a way for querying services of all datacenters? If I do not give any datacenter, always the local datacenter is used. In my case I need the services from all datacenters. Is there a simple way to do this with CT?

[slackpad]

Hi @msvechla there's no first-class way to do this. You range of the list of datacenters via {{ datacenters }} and feed that into a service query, though.

[aaronhurt]

@msvechla you probably already figured it out but it takes a bit more finagling to actually get what you want. Something like this should work ... the real trick is the printf to get the format of the datacenter name into something that the services call will accept.

{{range datacenters -}}
    {{- $dcOpt := printf "@%s" . -}}
    {{- range services $dcOpt -}}
        {{.Name}}{{$dcOpt}}{{"\n"}}
    {{- end -}}
{{end}}

[kak-tus]

@slackpad @sethvargo Hello, can I able to use consul-template now with prepared queries? Main problem is to get port of service in template (ip I can get thru dns with some_service.query.consul).

[jippi]

@slackpad is there anything in this ticket i can help with? it's something I would love to use within Nomad template{} - is it blocked on upstream features in Consul or system design in CT ?

[slackpad]

Hi @jippi,

For my personal case, a per-entry (maybe?) interval for refreshing

That approach looks pretty good for this. @sethvargo what do you think?

[wryfi]

Is anyone working on this? It would be very nice to have in my environment.