chaoss / grimoirelab-elk

GNU General Public License v3.0
58 stars 121 forks source link

[enrich/demographics] Fix fetching authors with min and max dates #1059

Closed zhquan closed 2 years ago

zhquan commented 2 years ago

This code fixes fetching all authors with min and max dates. Currently, ElasticSearch and OpenSearch use composite aggregation to paginate all buckets.

The old query only returns 10000 items and also returns the following warning message:

Deprecation: This aggregation creates too many buckets (10001) and
will throw an error in future versions. You should update the
[search.max_buckets] cluster setting or use the [composite]
aggregation to paginate all buckets in multiple requests.

Tests added acconrdingly.

Signed-off-by: Quan Zhou quan@bitergia.com

coveralls commented 2 years ago

Pull Request Test Coverage Report for Build 2434201959


Files with Coverage Reduction New Missed Lines %
/home/runner/work/grimoirelab-elk/grimoirelab-elk/grimoire_elk/enriched/enrich.py 46 73.16%
<!-- Total: 46 -->
Totals Coverage Status
Change from base Build 2434131458: 0.04%
Covered Lines: 8841
Relevant Lines: 10731

💛 - Coveralls
jjmerchante commented 2 years ago

LGTM