apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.63k stars 3.56k forks source link

GH-44800: [C#] Implement Flight SQL Client #44783

Open HackPoint opened 3 days ago

HackPoint commented 3 days ago

GH-44800: [C#] Implement Flight SQL Client

Rationale for this Change

This pull request introduces a new implementation of FlightSqlClient and PreparedStatement in C#. Previously, there was no C# client for Flight SQL, leaving a significant gap for .NET developers who wished to interact with Flight SQL servers.

The implementation aligns with the existing C++ Flight SQL client API, ensuring consistent and familiar behavior across languages and providing a robust client for the Apache Arrow ecosystem in .NET.


What's Included in this PR?

Key Features

  1. FlightSqlClient:

    • Provides query execution (ExecuteAsync, ExecuteUpdateAsync) and schema retrieval (GetCatalogsAsync, GetDbSchemasAsync, etc.).
    • Implements metadata operations for catalogs, schemas, tables, and more.
    • Fully integrated with gRPC and Apache Arrow ecosystems.
    • Supports extensibility for advanced features like transactions.
  2. PreparedStatement:

    • Implements parameterized query execution (SetParameters, ExecuteAsync, and ExecuteUpdateAsync).
    • Supports lifecycle management (CloseAsync) for effective resource handling.
    • Aligns with the prepared statement design in the C++ client.

API Consistency

This implementation mirrors the C++ Flight SQL client to ensure API alignment across supported languages:


Are These Changes Tested?

Testing Overview

  1. Unit Tests:

    • Added tests for query execution and parameter binding in PreparedStatement.
    • Verified schema retrieval methods like GetCatalogsAsync and GetDbSchemasAsync.
  2. Integration Tests:

    • Tested against a live Flight SQL server to validate query execution, schema retrieval, and metadata operations.
  3. End-to-End Tests:

    • Covered real-world scenarios for parameterized updates and queries, ensuring robustness.

Example Test Cases


Are There Any Breaking Changes?

This PR introduces new functionality and does not affect any existing features. There are no breaking changes.


Are There Any User-Facing Changes?

New Capabilities

  1. FlightSqlClient:

    • Query execution and schema retrieval for SQL queries on Flight SQL servers.
    • Metadata retrieval for catalogs, schemas, tables, and more.
  2. PreparedStatement:

    • Supports parameterized queries with proper parameter binding.
    • Provides robust lifecycle management and execution.

API Consistency


Additional Notes


Resources


Feedback and Suggestions

Thank you for reviewing this contribution! Suggestions and feedback are welcome to ensure the implementation meets the project's standards and requirements.

github-actions[bot] commented 3 days ago

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

CurtHagenlocher commented 3 days ago

Thanks for a very large change! Could you please add the official Apache header to all the source files and open an issue to link this to?

HackPoint commented 2 days ago

Thanks for a very large change! Could you please add the official Apache header to all the source files and open an issue to link this to?

Sure, sir, I will do everything mentioned here. Thank you for your guidelines.

github-actions[bot] commented 1 day ago

:warning: GitHub issue #44800 has been automatically assigned in GitHub to PR creator.

assignUser commented 1 day ago

I approved the workflows.

I don't work on C# sharp but wanted to thank you @HackPoint for such a big first-time contribution!