prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
16.07k stars 5.38k forks source link

[native] Add Arrow Flight Connector #24082

Open Rijin-N opened 5 days ago

Rijin-N commented 5 days ago

Description

Add Prestissimo support for Arrow Flight connectors.

The native Arrow Flight connector can be used to connect to any Arrow Flight enabled Data Source. The metadata layer is handled by the Presto coordinator and does not need to be re-implemented in C++. Any Java connector that inherits from presto-base-arrow-flight can use this connector as it's counterpart for the Prestissimo layer.

Different Arrow-Flight enabled data sources can differ in authentication styles. A plugin-style interface is provided to handle such cases with custom authentication code.

Motivation and Context

RFC: https://github.com/prestodb/rfcs/blob/main/RFC-0004-arrow-flight-connector.md#prestissimo-implementation

Impact

Arrow Flight based connectors can be used with Prestissimo

Test Plan

Unit Tests set up a mock arrow flight server and exchange data using the new connector.

Contributor checklist

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

Prestissimo (Native Execution)
* Add support for Arrow Flight connectors :pr:`24082`
linux-foundation-easycla[bot] commented 5 days ago

CLA Missing ID CLA Not Signed

Rijin-N commented 5 days ago

This PR requires a velox support PR https://github.com/facebookincubator/velox/pull/11588, to install Arrow Flight dependencies.

prestodb-ci commented 4 days ago

Saved that user @Rijin-N is from IBM