search

spotify / heroic

The Heroic Time Series Database
https://spotify.github.io/heroic/
Apache License 2.0
848 stars 109 forks source link

Dynamic Metadata Read Indices #746

Closed adsail closed 3 years ago

adsail commented 3 years ago

Introduce support for RotatingIndexMapping to limit the number of elasticsearch metadata indices read based upon the range of a heroic query. If the newly introduced config flag is set to true, the number of read indices is dynamically determined based upon the rotation interval and the range of the query.

This PR also makes the non-dynamic maxReadIndices configurable so as to properly support delete. This variable should be set to the maximum number of queryable indices that could possibly exist in Elasticsearch based upon the rotation interval and ILM policy.

Closes #745.

codecov[bot] commented 3 years ago

Codecov Report

Merging #746 (e8c0701) into master (a055e34) will decrease coverage by 0.07%. The diff coverage is 34.48%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #746      +/-   ##
============================================
- Coverage     55.08%   55.01%   -0.08%     
- Complexity     3145     3149       +4     
============================================
  Files           746      746              
  Lines         20334    20377      +43     
  Branches       1329     1336       +7     
============================================
+ Hits          11201    11210       +9     
- Misses         8652     8680      +28     
- Partials        481      487       +6
Impacted Files Coverage Δ Complexity Δ
...y/heroic/profile/ElasticsearchMetadataProfile.java 9.09% <0.00%> (-1.26%) 2.00 <0.00> (ø)
...y/heroic/elasticsearch/index/SingleIndexMapping.kt 0.00% <ø> (ø) 0.00 <0.00> (ø)
...roic/metadata/elasticsearch/MetadataBackendKV.java 74.06% <33.33%> (-6.61%) 50.00 <4.00> (ø)
...heroic/elasticsearch/index/RotatingIndexMapping.kt 70.96% <46.66%> (-7.76%) 16.00 <3.00> (+3.00) :arrow_down:
...com/spotify/heroic/aggregation/simple/MinBucket.kt 55.55% <0.00%> (+11.11%) 5.00% <0.00%> (+1.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a055e34...e8c0701. Read the comment docs.