Provider finally broke with Elastic 8.6.0

[bplies-ATX]

This provider has worked well for us up through 8.5.3 but now is failing in 8.6.0 for inexplicable HTTP 401 reasons.

Error: elastic: Error 401 (Unauthorized)

  with elasticsearch_xpack_index_lifecycle_policy.ilm_functionbeat_policy,
  on elastic.tf line 104, in resource "elasticsearch_xpack_index_lifecycle_policy" "ilm_functionbeat_policy":
 104: resource "elasticsearch_xpack_index_lifecycle_policy" "ilm_functionbeat_policy" {

Error: elastic: Error 401 (Unauthorized)

  with elasticsearch_component_template.idx_comp_templ_ecs_agent,
  on elastic.tf line 112, in resource "elasticsearch_component_template" "idx_comp_templ_ecs_agent":
 112: resource "elasticsearch_component_template" "idx_comp_templ_ecs_agent" {

The credentials hadn't changed. There is no explicit mention of any breaking changes in ES 8.6.0 https://www.elastic.co/guide/en/elasticsearch///reference/master/migrating-8.6.html#breaking-changes-8.6 https://www.elastic.co/guide/en/elasticsearch///reference/master/release-notes-8.6.0.html#release-notes-8.6.0

@olivere, the maintainer, of the underling Go library utilized by this provider states

Notice however that with ES 8.x, you need to use the official Go client.

There was also discussion of adopting the REST API Compatibility feature of Elastic. But who knows when that'll happen.

Sadly, it is looking like this provider may be reaching a dead end (with Elastic at least, not OpenSearch). It has provided a vital service in the absence of an official provider, but now Elastic seems to be attempting to finally offer one. I suppose that is the proper way to go, but I wish it were as battle-hardened as this one.

[phillbaker]

Hello - please see https://github.com/phillbaker/terraform-provider-elasticsearch/issues/268, unfortunately the official client is the way to go for ES v8

[bplies-ATX]

Just a courtesy follow-up. We discovered that this provider does still work. There was an unrelated bug with Terraform and AWS Secret Manager that manifested when changing tags on the Secret resource. For some reason that interfered with using the secrets that this provider needed to interact with Elastic. Hence the 401/Unauthorized errors.

This benign-looking change caused a problem.

   tags = {
     Application = local.elastic_service_name
     Environment = local.elastic_service_env
-    team        = local.elastic_team
+    Squad       = local.elastic_team
+    ServiceEnv  = local.elastic_service_env
   }
 }