elastic / terraform-provider-elasticstack

Terraform provider for Elastic Stack
https://registry.terraform.io/providers/elastic/elasticstack/latest/docs
Apache License 2.0
170 stars 92 forks source link

[Bug] Error Creating Elasticsearch Index with Date in Name via Terraform Provider Due to Unsupported Special Characters #882

Open 01martinelli opened 1 week ago

01martinelli commented 1 week ago

Description When trying to create an index with a date in the name via the Terraform Provider for Elastic, the operation fails due to unsupported special characters in the index name. The same command works in Kibana DevTools but not through the Terraform provider.

To Reproduce Steps to reproduce the behavior:

  1. Terraform (TF) configuration used:
    resource "elasticstack_elasticsearch_index" "nginx-k8s-prod-logs-vcae" {
     name = " %3Cnginx-k8s-prod-logs-vcae-%7Bnow%2Fs%7Byyyy-MM-dd%7D%7D-00001%3E"
     aliases = {
       "nginx-k8s-prod-logs-vcae" = {
         is_write_index = true
       }
     }
    }
  2. Execute the following TF operations to replicate the error:
    • terraform plan
    • terraform apply
  3. Observe the following error in the output:
    ╷
    │ Error: invalid value for name (must contain lowercase alphanumeric characters and selected punctuation, see: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#indices-create-api-path-params)
    │
    │ with elasticstack_elasticsearch_index.nginx-k8s-prod-logs-vcae,
    │ on rollover-index_vcae.tf line 2, in resource "elasticstack_elasticsearch_index" "nginx-k8s-prod-logs-vcae":
    │ 2: name = "nginx-k8s-prod-logs-vcae-{now/s{yyyy-MM-dd}}-00001"
    │

Expected behavior The index should be created successfully with the specified date in the name, similar to how the command works when executed directly in Kibana DevTools.

tobio commented 1 week ago

This is an interesting issue with the provider. As it's written, supporting this behaviour doesn't really make sense since {now/s{yyyy-MM-dd}} would evaluate to something different if the TF module was applied on different days. The provider needs to track the initial index, not just a random one matching the pattern.

Fixing this issue in isolation will then mean users hit https://github.com/elastic/terraform-provider-elasticstack/issues/94. There's a larger body of work required here to properly support lifecycle policy managed groups of indices within the provider. On the surface there's likely diminishing value in adding that support over say using data streams instead (I may certainly be missing some common cases there though).