elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.22k stars 24.85k forks source link

Remove limitation of using full text functions after LIMIT #115297

Open ioanatia opened 3 weeks ago

ioanatia commented 3 weeks ago

When using match or qstr after limit we are returning an error that indicates this is not currently supported:

FROM kibana_sample_data_logs
| LIMIT 10000 
| WHERE MATCH(geo.dest, \"CN\")

In Kibana, in order to suggest values for filters, we construct a query similar to:

FROM kibana_sample_data_logs
| LIMIT 10000
| WHERE MATCH(geo.dest, \"CN\")
| WHERE `geo.src` IS NOT NULL
| STATS `geo.src_terms` = count(`geo.src`) BY `geo.src`
| SORT `geo.src_terms` DESC\n 
| LIMIT 10

This query would return suggested values for the geo.src field:

Image

Because of the limitation we have for match where it cannot be used after limit, Kibana will return an error when it tries to suggest values for filters:

Image

related https://github.com/elastic/kibana/issues/196995

Kibana is one use case that would benefit from removing this limitation. We can expect that ES|QL users will also hit it in other contexts.

elasticsearchmachine commented 3 weeks ago

Pinging @elastic/es-search-relevance (Team:Search Relevance)

elasticsearchmachine commented 3 weeks ago

Pinging @elastic/kibana-esql (ES|QL-ui)

ChrisHegarty commented 3 weeks ago

For now, and until this issue is resolved, the match and qstr functions will be held to snapshot builds in 8.x, see #115253.