search

StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.32k stars 1.68k forks source link

add Thrift SQL API #38958

Open mingmxu opened 6 months ago

mingmxu commented 6 months ago

Feature request

Is your feature request related to a problem? Please describe. The HTTP SQL API use HTTP/JSON for data transportation, it's not efficient to query/load large dataset. In the other hand, HTTP has weak constrain on input/output typing, more error prone for integration especially during upgrade.

Describe the solution you'd like I would like to introduce a Thrift API along with(or replace) the HTTP API, as a full-function alternative of the JDBC protocol.

Describe alternatives you've considered gPRC+protobuf is another option. Since Thrift is used already for fe-be communication, it's better to stick with single solution.

Additional context We would like to move away from the JDBC protocol, to find another option which is better to integrate with internal observability, service discovery etc.

imay commented 6 months ago

@mingmxu Actually, on our roadmap, we want to support Arrow Flight. But haven't started. So if you want to contribute to enhancing the result output capability, I suggest you work on it.

mingmxu commented 6 months ago

Please correct me if wrong, is Arrow Flight more than an alternative of the JDBC API, rather than the HTTP API?

Edit, from what I read, Arrow Flight is a stateful, long-live client/server model, vs stateless in Thrift/HTTP.

imay commented 6 months ago

Yes you are right, this is an alternative of JDBC. Can I say that what you want is a stateless api?

mingmxu commented 6 months ago

Can I say that what you want is a stateless api?

Yes, we want to find a replacement for HTTP SQL API introduced in version 3.2. Currently we’re using JDBC but it makes StarRocks a black-box when integrating with datadog tracing, the trace context cannot pass through.

kimmking commented 6 months ago

@imay hi, imay, how to see the SR roadmap. thx.