Unable to configure the total field limit for an elasticsearch index

[JesseDavda]

I'm using Terraform v1.3.5 and provider version v2.0.7. I have a large index (over 1000 fields) that fails to create with the error:

Error: elastic: Error 400 (Bad Request): Limit of total fields [1000] has been exceeded [type=illegal_argument_exception]

As far as I can see there are no arguments in either the elasticsearch_index or the elasticsearch_cluster_settings resources that allow configuration of this value.

My resource Terraform code:

resource "elasticsearch_index" "large_index" {
  name               = "large-index"
  number_of_shards   = 1
  number_of_replicas = 2
  mappings           = file("${path.root}/elastic-search-mappings/large-index.json")
  analysis_analyzer  = jsonencode({
    email_analyzer = {
      tokenizer = "uax_url_email"
    }
  })
}

[phillbaker]

Hi @JesseDavda can you link to the documentation for the specific index setting that would control this?

[JesseDavda]

Hi @phillbaker this is a link to an elastic search forum post regarding increasing the field limit. I haven't been able to find it in their official documentation, but I can confirm that this setting is accepted and does increase the field limit

[sergiojoker11]

What I haven't been able to achieve is to import an existing index and then increasing this sort of limit. The behaviour I get is a "force replacement" on the mappings resource field. Steps: 1) Index is successfully imported with this command:

tf import module.audit_trail_service_v2.elasticsearch_index.audit-trail-index flexys-perftest-events-audit-trail-service

2) then when I tf apply the following:

resource "elasticsearch_index" "audit-trail-index" {
  name               = "flexys-perftest-events-audit-trail-service"
  number_of_shards   = 1
  number_of_replicas = 2
  mappings           = <<EOF
  {
    "nested_objects": {
      "limit": "10000"
    },
    "total_fields" : {
      "limit" : "10000"
    }
  }
  EOF
}

3) it tries a force replacement

Any pointers @phillbaker ?

[JesseDavda]

@phillbaker bump, this is a blocker for me implementing your provider in production. Any updates?