apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
https://kyuubi.apache.org/
Apache License 2.0
2.11k stars 914 forks source link

[FEATURE] Support SparkSQL on Standalone cluster #6716

Open nqvuong1998 opened 1 month ago

nqvuong1998 commented 1 month ago

Code of Conduct

Search before asking

Describe the feature

Kyuubi currently provides an excellent interface for running Spark SQL on distributed Spark environments. However, it lacks official support for running SparkSQL on a standalone Spark cluster. This feature request proposes adding support for SparkSQL execution on Spark Standalone clusters, allowing users to deploy Kyuubi with more flexibility in Spark cluster configurations.

Motivation

No response

Describe the solution

No response

Additional context

No response

Are you willing to submit PR?

nqvuong1998 commented 1 month ago

cc @yaooqinn @pan3793

yaooqinn commented 1 month ago

Technically, Standalone clusters are supported. What was the issue you actually encountered?

nqvuong1998 commented 1 month ago

Hi @yaooqinn , Apache Spark Kubernetes Operator supports deploying a Spark Standalone cluster on Kubernetes. However, I have come across an issue where Kyuubi has not yet implemented a Spark Standalone ApplicationOperation. Therefore, I created this feature request.

yaooqinn commented 1 month ago

AFAIK, ApplicationOperation is irrelevant to submit engines