[ANN] Upcoming changes to Consul Provider

[pearkes]

In order to be transparent with the roadmap and broadcast some substantial changes coming to the provider, I'm opening this issue in the hopes that interested users will see it. Any changes will be versioned and included in changelogs, with old versions continuing to work (as long as the upstream Consul API continues to support them). You can learn more about pinning a provider version here.

In time I will create milestones within GitHub with some stand-alone issues for each of the below changes. Timeline across this will likely be flexible -- any help is welcome, just feel free to make an comment here.

These changes are primarily to simplify the number of resources, ensure we are using the correct APIs/design provided by Consul for something like Terraform, and remove resources that can no longer be supported by the current version of the Consul API.

Existing Resources

Name	Type	Action	Version
consul_agent_self	Datasource	Deprecate/Remove	2.0.0/3.0.0
consul_agent_config	Datasource	Create	2.0.0
consul_catalog_nodes	Datasource	Rename (consul_nodes)	2.0.0
consul_catalog_services	Datasource	Rename (consul_services)	2.0.0
consul_catalog_service	Datasource	Rename (consul_service)	2.0.0
consul_node	Datasource	Create	2.x.0
consul_keys	Datasource	No change	n/a
consul_agent_service	Resource	Deprecate/Remove	2.0.0/3.0.0
consul_catalog_entry	Resource	Deprecate/Remove	2.0.0/3.0.0
consul_keys	Resource	No change	n/a
consul_key_prefix	Resource	No change	n/a
consul_node	Resource	No change	n/a
consul_service	Resource	Use catalog API	n/a

New Resources

These are resources that we intend to add in the future.

Name	Type	Version
consul_autopilot_health	Datasource	2.x.0
consul_autopilot_config	Resource	2.x.0
consul_network_segments	Resource	3.x.0
consul_network_area	Resource	3.x.0
consul_network_area_members	Datasource	3.x.0
consul_acl_token*	Datasource	3.x.0

* This assumes that the Consul API supports some notion of a "token accessor", otherwise I don't think it would be wise to support it given the security implications.

[vchan2002]

Thanks for the FYI in regards to this...

I was about to raise a question in regards to the difference in between consul_service and consul_catalog_entry; they appear to do the same thing... In fact, the documentation recommends that consul_service be not used in lieu of consul_catalog_entry.

Yet, I see that consul_catalog_entry is on the chopping block... Does that mean the documentation, as of now, is not accurate? Just wondering as we're looking to leverage some of these resources.

[pearkes]

@vchan2002 Yeah great question -- you're right the current consul_service does the wrong thing.

This plan does actually involve changing the consul_service resource -- with an attempt to make it use the catalog APIs as it should. I'm going to see if there is a backwards compatible way to achieve that (currently have not investigated). I've updated the table above to be more clear about that.

The current consul_service resource uses the wrong endpoints (local agent) for registering external services, and it doesn't really make sense in Terraform. Here's where the incorrect API is being used in consul_service:

https://github.com/terraform-providers/terraform-provider-consul/blob/58acfd5a3e2e109666b3f51de0c58c0fa1907cfe/consul/resource_consul_service.go#L84-L86

The consul_catalog_entry resource does to it properly:

https://github.com/terraform-providers/terraform-provider-consul/blob/58acfd5a3e2e109666b3f51de0c58c0fa1907cfe/consul/resource_consul_catalog_entry.go#L177-L180

However it uses this nested service {} block (to allow nodes to be registered as well). So, neither of them do it truly how we want the UX to be -- for someone registering an external service (sensible and common use-case) we want to provide an obvious resource to do that with in the provider, which is not currently the case. Depending on what kind of migration we can accomplish, the backup plan would be to deprecate it all in favor of consul_external_service and rename it to consul_service at some future time.

Curious thoughts on that if you have any!

[tecnobrat]

Does this mean support for health checks are coming?

[pearkes]

@tecnobrat Not planned as part of this, but can you maybe open a separate issue on this repo that explains the use case and some examples?

[vchan2002]

I guess, as a followup....

At this time, If I were to want to setup a external consul service today, which provider should I use that would be the least painful in terms of forwards compatibility, when this is redone? Should I use consul_service, even though it's not doing 100% the right thing, as you said?

[pearkes]

@vchan2002 consul_catalog_entry is using same APIs and similar behavior as the upcoming version of consul_service, but in some cases it won't matter either way. consul_catalog_entry will require more modifications to the configuration, but you can be sure it will be doing the same thing behind the scenes (including requiring you to specify a node, unlike consul_service).

[sriyer]

@pearkes we are one of the enterprise users of consul 1.1.0 and we use the provider on our automation setup. We've enhanced it to add network area and acl token creation and update support, we are still sorting through the process on our side to contribute back though.

we'd love to hear on the implementation directions that have been thought of on these two resources and data sources so we could make sure our's is in congruence with what's planned.

cc: @samart

[pearkes]

@sriyer Would happily discuss my point of view and hear about the approach you took. If you could upstream it even better. I haven't gotten to thinking about implementation for both of those resources yet. The ACL management is far trickier given the simplistic nature of the ACL system in Consul -- there are definite security challenges that could not be worked around easily based on how the system works today. I am aware of some planned improvements to the Consul ACL system that would obviate that, however.

If you want to talk privately, feel free to use my email on my GitHub profile and we can connect to discuss further?