search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.22k stars 437 forks source link

[GLUTEN-6920][CORE] Move API `Backend#defaultBatchType` down to `BackendSettingsApi` in module gluten-substrait #8016

Open zhztheplayer opened 20 hours ago

zhztheplayer commented 20 hours ago

Part of https://github.com/apache/incubator-gluten/issues/6920.

defaultBatchType was designed as a global value for query planner to:

  1. Decide which type of columnar batch a unrecognizable operator will output (e.g., AdaptiveSparkPlanExec, ColumnarBroadcastExchangeExec, etc.)
  2. Decide which type of columnar batch a Substrait operator will output (e.g., xxxTransformer)

In regard of mixing different backends, we'd move this global value down to SubstraitBackend API, which is designed to have only one instance loaded at a time so there will be no conflicts on this value among all loaded backends. This is to keep APIs in gluten-core simple for extensibility purpose.

github-actions[bot] commented 20 hours ago

Run Gluten Clickhouse CI on x86

github-actions[bot] commented 20 hours ago

https://github.com/apache/incubator-gluten/issues/6920

github-actions[bot] commented 2 hours ago

Run Gluten Clickhouse CI on x86