microsoft / hyperspace

An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.
https://aka.ms/hyperspace
Apache License 2.0
423 stars 115 forks source link

[PROPOSAL]: ZOrderCoveringIndex #515

Open sezruby opened 2 years ago

sezruby commented 2 years ago

NOTE: This is an issue for tracking Z-order covering index.

The initial implementation is ready in #495.

Problem Statement

Currently, CoveringIndex data is stored as bucketed, and it just sorted within a bucket. So it is usually less effective for filter queries compared to fully sorted data. In addition, the index is applied when a given query has conditions only with the first indexed column.

Background and Motivation

TBD

Proposed Solution

Introduce Z-order covering index type to optimize filter query for multiple columns. TBD

Known/Potential Compatibility Issues

TBD

Design

TBD

Implementation