ESQL: Allow access to failure store

[flash1293]

Description

The failure store is a very new experimental data stream feature that captures documents that couldn't be indexed and stores them in a special index with a fixed mapping in the failure store.

Certain key APIs like search or field caps gain a new argument failure_store, which can be used to include it into queries:

GET logs-mystream-prod/_search?failure_store=only
GET logs-mystream-prod/_search?failure_store=include
GET logs-mystream-prod/_search?failure_store=exclude # default

This allows users to troubleshoot ingestion problems and possibly even re-index failed documents after the fact.

As it needs to be possible to browse the documents in the failure store, it should be possible to query it from ESQL to be able to use all the regular tooling (Discover, Alerts, ...)

Some unstructured thoughts how that could work

I'm not sure about the right approach, but this is something that will definitely come up, as we plan to integrate the failure store in various places to make it easy for users to mitigate ingestion issues.

• Directly query the backing index: FROM .fs-logs-mystream-prod-0001 | where error=="pipeline" .. (pretty clunky)
• Add the failure_store param to the ESQL REST endpoint as well (might be hard to use in some cases)
• Add a special ESQL keyword, e.g. FROM logs-mystream-prod FAILURE_STORE only | ...

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[flash1293]

cc @jbaiera - there might have been discussions around this already

[flash1293]

cc @yngrdyn @davismcphee - I think this will become relevant soon

[nik9000]

Merging the failure store documents with the successful ones sounds like it'd change the shape of the output quite a bit and make ESQL tricky to use. Not impossible at all, but tricky.

[flash1293]

FWIW, limiting the choice to only and exclude would make sense to me as well