opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.76k stars 1.81k forks source link

[BUG] `doc_count_error_upper_bound` for terms agg should not show 0 when descending _count sort is not used #11851

Open jed326 opened 10 months ago

jed326 commented 10 months ago

Describe the bug

Encountered this as a part of https://github.com/opensearch-project/OpenSearch/pull/11732

doc_count_error is only relevant when the buckets are sorted in descending _count order (I think this is a documentation gap too, but that's a separate issue https://opensearch.org/docs/latest/aggregations/bucket/terms/). If a different sort order is used then doc_count_error_upper_bound currently shows 0. This can be confusing for users since it implies that there are no candidate buckets being eliminated by shard_size.

Related component

Search:Aggregations

To Reproduce

Example using noaa data from OSB but the underlying data doesn't really matter:

GET weather-data-2016-multi/_search
{
  "size": 0,
  "aggs": {
    "station": {
      "terms": {
        "field": "station.elevation",
        "size": 1
        , "order": {
          "_key": "asc"
        }
      }
    }
  }
}

{
  "took": 1256,
  "timed_out": false,
  "terminated_early": true,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "station": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 33581715,
      "buckets": [
        {
          "key": -999.9000244140625,
          "doc_count": 77766
        }
      ]
    }
  }
}

Expected behavior

Whenever the doc_count_error cannot be calculated, it either show not be returned in the response body or it should return -1 to differentiate the cases where there is no error vs cases where it cannot be computed.

Additional Details

Plugins Please list all plugins currently enabled.

Screenshots If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

Additional context Add any other context about the problem here.

peternied commented 9 months ago

[Triage - attendees 1 2] @jed326 Thanks for filing