Open shatp opened 2 years ago
I am experiencing the same issue on opensearch version 2.9. For example when running this query in opensearch dashboard:
{
"query": {
"bool": {
"filter": [
{
"term": {
"tenant_id": 1
}
},
{
"term": {
"user_id": "user-34"
}
},
{
"range": {
"event_time_utc_ms": {
"gte": 1694171042000
}
}
}
]
}
},
"aggs": {
"dst_type": {
"terms": {
"field": "dst_type"
}
},
"dst_category_name": {
"terms": {
"field": "dst_category_name"
}
}
}
}
I see this in the slow logs. Notice the null range filter:
"query": {
"bool": {
"filter": [
{
"term": {
"tenant_id": {
"value": 1,
"boost": 1
}
}
},
{
"term": {
"user_id": {
"value": "user-34",
"boost": 1
}
}
},
{
"range": {
"event_time_utc_ms": {
"from": null,
"to": null,
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
}
So, the problem seems to arise from the fact that slow logs are written at the shard level and queries can be rewritten at the shard level.
Specifically, there is logic in RangeQueryBuilder
shard-level rewrite that checks if the current shard is entirely contained within the range. If so, it replaces the upper and lower bounds with null
, which functionally turns it into a "does the field exist?" query: https://github.com/opensearch-project/OpenSearch/blob/966b78dae39fd3538d78f05bdc2b5fbdc1e1301f/server/src/main/java/org/opensearch/index/query/RangeQueryBuilder.java#L486-L493
I can see a few possible options, all of which require changes in OpenSearch (i.e. I see no user workaround):
match_none
.) RangeQueryBuilder
to keep the endpoints, but still allow it to execute as a FieldExistsQuery
. (I would probably add a flag to RangeQueryBuilder
that only gets set on rewrite, and essentially says "run as a FieldExistsQuery
".)
Describe the bug Looking at the slow query logs from OpenSearch, we see that the range -> from/to values are null when queries are run from Quick select of Discover Tab.
To Reproduce Steps to reproduce the behavior:
"filter": [ { "range": { "@timestamp": { "from": "2022-02-02T16:03:10.792+05:30", "to": "2022-02-03T16:03:10.792+05:30", "include_lower": true, "include_upper": false, "boost": 1 } } },