Async ES|QL Query API

[ChrisHegarty]

Description

Add an Async Query API to ES|QL.

Proposed REST endpoints:

1. GET /_query/async
2. GET /_query/async/{id}
3. DELETE /_query/async/{id}
4. ~GET /_query/async/status/{id}~

, which complement the existing sync endpoint; GET /_query

Tasks

• [x] Enhance transport action to be async - sync can be built atop
• [x] Add public REST API to initiate async query https://github.com/elastic/elasticsearch/pull/103398
• [x] Add public REST API to GET async query results https://github.com/elastic/elasticsearch/pull/103398
• [x] Add public REST API to DELETE/CANCEL async query results https://github.com/elastic/elasticsearch/pull/103628
  • [x] Ensure that cancellation actually cancels the running query.
  • [x] Delete the stored results
• [x] ~Add public REST API to retrieve query status~ This is not needed by Kibana, defer
• [x] Enhance documentation for the new endpoints #104054
• [x] Improve test coverage between sync and async APIs #103700
• [x] Test that verifier and parser exceptions are serializable
• [x] Ensure that warnings are captured correctly #103700
• [x] Add tests to assert security privileges #104137
• [x] Run yaml tests async #104205
• [ ] Run yaml tests async in mixed version cluster #104206
• [x] #104379

[elasticsearchmachine]

Pinging @elastic/es-ql (Team:QL)

[elasticsearchmachine]

Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL)

[elasticsearchmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)