elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.82k stars 24.7k forks source link

Include is_write_index for data streams in _alias and _cat/aliases APIs #86633

Open rseldner opened 2 years ago

rseldner commented 2 years ago

Description

Currently, to determine the current write backing index of a datastream, I think one must assume it is the last index in a datastream's indices array in the GET datastream API.

...
      "name" : ".fleet-actions-results",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-.fleet-actions-results-2022.01.11-000003",
          "index_uuid" : "HffF2op6S8avpK_RAbPRMA"
        },
        {
          "index_name" : ".ds-.fleet-actions-results-2022.02.10-000004",
          "index_uuid" : "wmHY5iuHRVWKo1YodZEjTQ"
        },
        {
          "index_name" : ".ds-.fleet-actions-results-2022.03.12-000005",
          "index_uuid" : "mm2m-It1SHy_3_kydBv-sA"
        },
        {
+          "index_name" : ".ds-.fleet-actions-results-2022.04.11-000008",
          "index_uuid" : "EsSDlS_wQlmG_cEiE7q_hA"
        }
      ]
+      "generation" : 8
...

One could also assume it's the highest generation/numerical suffix when comparing index names GET _cat/indices?s=index

.ds-.fleet-actions-results-2022.01.11-000003
.ds-.fleet-actions-results-2022.02.10-000004
.ds-.fleet-actions-results-2022.03.12-000005
+.ds-.fleet-actions-results-2022.04.11-000008

This is tricky when attempting to accurately, and programmatically, determine the current write index.

It would be helpful to simply include data stream indices in the GET _alias and GET _cat/aliases APIs and indicate which is the current write index with is_write_index: true/false as we currently see with traditional indices.

GET _alias/.alerts-observability.metrics.alerts-default

{
  ".internal.alerts-observability.metrics.alerts-default-000001" : {
    "aliases" : {
      ".alerts-observability.metrics.alerts-default" : {
        "is_write_index" : false
      }
    }
  },
  ".internal.alerts-observability.metrics.alerts-default-000002" : {
    "aliases" : {
      ".alerts-observability.metrics.alerts-default" : {
        "is_write_index" : false
      }
    }
  },
  ".internal.alerts-observability.metrics.alerts-default-000003" : {
    "aliases" : {
      ".alerts-observability.metrics.alerts-default" : {
        "is_write_index" : false
      }
    }
  },
  ".internal.alerts-observability.metrics.alerts-default-000006" : {
    "aliases" : {
      ".alerts-observability.metrics.alerts-default" : {
+        "is_write_index" : true
      }
    }
  },
  ".internal.alerts-observability.metrics.alerts-default-000005" : {
    "aliases" : {
      ".alerts-observability.metrics.alerts-default" : {
        "is_write_index" : false
      }
    }
  },
  ".internal.alerts-observability.metrics.alerts-default-000004" : {
    "aliases" : {
      ".alerts-observability.metrics.alerts-default" : {
        "is_write_index" : false
      }
    }
  }
}
GET _cat/aliases?v&h=alias,index,is_write_index&s=alias
alias                                                    index                                                        is_write_index
...
.alerts-observability.metrics.alerts-default             .internal.alerts-observability.metrics.alerts-default-000002 false
.alerts-observability.metrics.alerts-default             .internal.alerts-observability.metrics.alerts-default-000004 false
.alerts-observability.metrics.alerts-default             .internal.alerts-observability.metrics.alerts-default-000003 false
.alerts-observability.metrics.alerts-default             .internal.alerts-observability.metrics.alerts-default-000001 false
.alerts-observability.metrics.alerts-default             .internal.alerts-observability.metrics.alerts-default-000005 false
+.alerts-observability.metrics.alerts-default             .internal.alerts-observability.metrics.alerts-default-000006 true
...

I'm not interested in any of the specific indices in the examples above; I just arbitrarily chose them for demonstration purposes.

elasticmachine commented 2 years ago

Pinging @elastic/es-data-management (Team:Data Management)

elasticmachine commented 2 years ago

Pinging @elastic/es-data-management (Team:Data Management)