search

microsoft / hyperspace

An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.
https://aka.ms/hyperspace
Apache License 2.0
424 stars 115 forks source link

[FEATURE REQUEST]: Enable hyperspace with SparkSessionExtention #503

Closed paryoja closed 3 years ago

paryoja commented 3 years ago

Need to add a functionality to enable Hyperspace with SparkSessionExtention to avoid change in user code.

Acceptance criteria

Describe what you consider (e.g., functionality requirement) has to be true to mark this feature as complete

Additional context

Discussed in https://github.com/microsoft/hyperspace/discussions/499

Originally posted by **ashokblend** October 7, 2021 We have server code which connects to spark jdbc(thrift) server to execute sql. We can have batch jobs which can create index for our data lake. But we are not getting how to enable hyperspace for spark thrift server, where we do not have control in initialising sparksession. So question is how to enable hyperspace in spark thriftserver ?