_meta description not working for settings

[thekofimensah]

Describe the bug I'm having an issue where I'm unable to put the _meta.description field into my settings component template. I get this error:

│ Error: Unable to create component template
│ 
│   with elasticstack_elasticsearch_component_template.metricbeat_noisy-settings[0],
│   on main.tf line 90, in resource "elasticstack_elasticsearch_component_template" "metricbeat_noisy-settings":
│   90: resource "elasticstack_elasticsearch_component_template" "metricbeat_noisy-settings" {
│ 
│ Failed with:
│ {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"unknown
│ setting [index._meta.description] please check that any required plugins
│ are installed, or check the breaking changes documentation for removed
│ settings"}],"type":"illegal_argument_exception","reason":"unknown setting
│ [index._meta.description] please check that any required plugins are
│ installed, or check the breaking changes documentation for removed
│ settings"},"status":400}

I'm using TF cloud for this and it doesn't make complete sense because it says that index._meta.description isn't valid. But my _meta is not part of the index object. Why does it think that?

In my main.tf

resource "elasticstack_elasticsearch_component_template" "metricbeat_noisy-settings" {
  name = "metricbeat_noisy-settings"
  template {
    settings = jsonencode(var.metricbeat_noisy-settings)
  }
}

and my tfvars:

metricbeat_noisy-settings = {
 "_meta": {
    "description": "test, test"
  },
  "index.lifecycle.parse_origination_date" : false,
  "index.lifecycle.name" : "prod_metricbeat",
  "index" : {
    "mapping" : {
      "total_fields" : {
        "limit" : "10000"
      }
    },
    "sort.field" : "@timestamp",
    "sort.order" : "desc"
  }
}

Using elastic/elasticstack 0.3.3 and my elasticseach version is 7.17.5. That setting works when I apply directly to ELK, it only doesn't work when applying it with terraform.

[RobsonSutton]

@thekofimensah - Hey 👋 The issue is just where you're trying to set the _meta field. This is a field that you set within mappings templates as opposed to settings templates. If you take a look below I can add this field to the mappings part of a component template: But when attempting to do the exact same thing within the settings template it will throw an exception:

Docs - https://www.elastic.co/guide/en/elasticsearch/reference/7.17/mapping-meta-field.html

You should just be able to add this to the mappings parameter in the template block for your component template instead. It'll all be contained in the same component template so will do the same thing you're looking to do. Alternatively there is a metadata parameter within the resource (https://registry.terraform.io/providers/elastic/elasticstack/latest/docs/resources/elasticsearch_component_template), that will do the exact same thing 👍

[thekofimensah]

Thanks for the response. I'm trying to put the _meta field as a sibling to the template object.. i.e.

PUT _component_template/test
{
  "_meta": {
    "description": "Working test"
  },
  "template": {
    "settings": {
      "index": {
        "mapping": {
          "total_fields": {
            "limit": "10000"
          }
        },
        "sort.field": "@timestamp",
        "sort.order": "desc"
      },
      "number_of_replicas": 0,
      "number_of_shards": "2",
      "refresh_interval": "30s",
      "max_docvalue_fields_search": "200"
    }
  }
}

This works just fine on my side.

[RobsonSutton]

@thekofimensah - Yeah so this does not sit within the template part of the request, the equivalent of what you are attempting to do should be achieved similar to the below using the metadata field:

resource "elasticstack_elasticsearch_component_template" "metricbeat_noisy-settings" {
  name = "metricbeat_noisy-settings"

  metadata = jsonencode({
      "description" = "Working test"
    })

  template {
    settings = jsonencode(var.metricbeat_noisy-settings)
  }
}

[thekofimensah]

Oh ok, that then works. Thanks!