[Gold Standard] Initial Code showing Hyperspace Indexes with a sample query

microsoft / hyperspace

An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.

https://aka.ms/hyperspace

Apache License 2.0

424 stars 115 forks source link

[Gold Standard] Initial Code showing Hyperspace Indexes with a sample query #385

Open apoorvedave1 opened 3 years ago

apoorvedave1 commented 3 years ago

What is the context for this pull request?

Gold Standard: Hyperspace code for creation of index metadata files and its use in Plan Stability test Suite.

Parent Issue: #334
Dependencies: #377
Dependent PRs: #386

What changes were proposed in this pull request?

In this PR: TPCDSHyperspace suite extends PlanStabilitySuite

Ability to define indexes : indexname, tablename, indexed cols(comma-separated), included cols(comma-separated)
Create index metadata files (metadata files are enough for generating query plans)
A simple query with join between two tables on a key
Approved plans (golden files) for the simple query

Does this PR introduce any user-facing change?

How was this patch tested?

Unit tests

imback82 commented 3 years ago

Should we just have code update + one query example?

apoorvedave1 commented 3 years ago

Should we just have code update + one query example?

@imback82 removed dependencies coming from the dependency PR. now this one contains only relevant files.

imback82 commented 3 years ago

Can you check the test failures?

apoorvedave1 commented 3 years ago

Can you check the test failures?

@imback82 thank you, I also cleaned up some other parts of the code. Please take a look.

imback82 commented 3 years ago

One way is to diff the file and attach it to the PR?

apoorvedave1 commented 3 years ago

One way is to diff the file and attach it to the PR?

I added files for without hyperspace enabled and also with the diffs. Please let me know if we intend to keep them in the final version or is it just for the sake of validation in the PR.?

imback82 commented 3 years ago

I added files for without hyperspace enabled and also with the diffs. Please let me know if we intend to keep them in the final version or is it just for the sake of validation in the PR.?

This is just for the PR.