I need a function to check whether Spark actually used an index I intended to use.
It could be helpful to understand the behavior of hyperspace indexes and optimize the behavior of it.
Acceptance criteria
Describe what you consider (e.g., functionality requirement) has to be true to mark this feature as complete
[ ] Check whether a LogicalPlan has any use of any hyperspace indexes.
[ ] Given hyperspace index name, check whether a LogicalPlan has a use of the index.
[ ] Given hyperspace index name and version value, check whether a LogicalPlan has a use of the index with the version value.
Success criteria
Define any measurements that need to be performed
[ ] Functionality implemented
Additional context
In the DeltaLakeIntegrationTest.scala, it has isIndexVersionUsed and isIndexUsed for checking the test results. It could be better to expose the functions for the general use.
hs.explain(df) returns used indexes, but I think we need to improve the output result so that it shows the used log version & the number of times each index applied.
Feature requested
I need a function to check whether Spark actually used an index I intended to use. It could be helpful to understand the behavior of hyperspace indexes and optimize the behavior of it.
Acceptance criteria
Describe what you consider (e.g., functionality requirement) has to be true to mark this feature as complete
Success criteria
Define any measurements that need to be performed
Additional context
In the
DeltaLakeIntegrationTest.scala
, it hasisIndexVersionUsed
andisIndexUsed
for checking the test results. It could be better to expose the functions for the general use.