Initial full-text search in ES|QL

[ChrisHegarty]

Meta issue listing tasks and subtasks for consideration as part of the Initial Search in ES|QL initiative. Initially the features will be available only on snapshot builds.

• [x] Add support for MATCH operator: FROM employees | WHERE title MATCH "test": @ioanatia https://github.com/elastic/elasticsearch/pull/110971
  • equivalent to a MATCH query
  • add tests to map failures to push to Lucene - at this stage we do not aim for graceful failures, but only to map all the failure cases through tests
• [x] Add support for MATCH command: FROM employees | MATCH "query string": @carlosdelest
  • Initial work: https://github.com/elastic/elasticsearch/pull/111147
  • Follow up work: https://github.com/elastic/elasticsearch/issues/111487
  • initially this will just default to a query_string query
  • add tests to map failures to push to Lucene - the same for MATCH operator, we only need to map all failure cases through tests
• [x] Add graceful failures for MATCH operator and command:
  • these need to happen before we make any optimizations to the logical plan - we might need a new dedicated place to add these validations, which is why this is a separate issue
• [x] Add dataset for CSV testing for full text search
• [ ] Handle RENAME with MATCH operator https://github.com/elastic/elasticsearch/pull/110971#discussion_r1682591916
• [ ] Investigate how to add options to support parameters for operators/commands (check with analytical engine group)
• [ ] Add support for scoring: FROM employees METADATA _score | WHERE title MATCH "test" | SORT _score @tteofili
  • adds _score as a metadata attribute
  • when _score is present it will be altered by queries such as MATCH
• [ ] Add qstr function @carlosdelest #112590
• [ ] Add match function
• [ ] Remove match command

Enhacements:

• [ ] support for query params to accept arrays of values (maintains readability for knn)

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[ioanatia]

With https://github.com/elastic/elasticsearch/pull/110132 merged we should have the basic filtering case covered. I think it was the only piece that was missing. Let me know if there's something else needed, but I am marking that as completed.

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)