search

opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.7k stars 1.8k forks source link

[Star tree] Star tree test cases / bugs tracker #14813

Open bharath-techie opened 3 months ago

bharath-techie commented 3 months ago

Is your feature request related to a problem? Please describe

This will the single issue where we can track all star tree related test cases, edge cases, potential bugs etc as we develop the feature.

Describe the solution you'd like

Address them as part of tests

Related component

Indexing:Performance

Describe alternatives you've considered

No response

Additional context

No response

bharath-techie commented 2 months ago

Need to add a separate issue for tracking performance issues

But using this issue to track open items in case of performance cases / test cases for now :

  1. https://github.com/opensearch-project/OpenSearch/pull/14817/files#r1698084827 - Measure IO performance for mergeFiles logic in off heap star tree builder.
  2. Benchmark the number of nodes created in memory in off heap.
bharath-techie commented 2 months ago
  1. Add randomness to builder tests - data type of the dimensions and metrics should be random, range should be random, null,0 should be heavily skewed etc.
bharath-techie commented 2 months ago
  1. Measure the performance of on heap vs off heap and see if there are any scenarios on heap is helpful and also the tradeoffs in terms of higher memory utilization against performance gain.
bharath-techie commented 2 months ago

Bug : Handle docs with -1 as value : https://github.com/opensearch-project/OpenSearch/issues/15097