search

sxt-partners / arrow-ballista

Apache Arrow Ballista Distributed Query Engine
https://arrow.apache.org/datafusion
Apache License 2.0
1 stars 2 forks source link

FlightSQL Integration test #4

Open avantgardnerio opened 1 year ago

avantgardnerio commented 1 year ago

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

FlightSQL support has been added to Ballista, but unfortunately it is hard to test because the only way (I know how) is using a custom fork of the FlightSQL JDBC driver. In order to ensure regressions don't happen, it would be good to test in an automated way.

Describe the solution you'd like

Automated integration testing between Java/JDBC and Rust would be ideal, but would add a lot of complexity. Therefor, it would be good to add a Rust FlightSQL client -> Ballista FlightSQL server test. I would recommend starting with a manual JDBC test to see how it is supposed to work, then emulating that test in pure Rust. The code-under-test is located here.

Describe alternatives you've considered

  1. A java/rust integration test with docker-compose in CI
  2. Manual testing
avantgardnerio commented 1 year ago

This appears to be the Flight Client https://github.com/spaceandtimelabs/arrow-rs/blob/e5b9d05ec50807666efe401729708d53216d79fc/arrow-flight/src/arrow.flight.protocol.rs#L218

avantgardnerio commented 1 year ago

I think we want a client here: https://github.com/apache/arrow-rs/tree/master/arrow-flight/src/sql (and an automated test too!) (https://spaceandtimeworkspace.slack.com/archives/D03RMCRMTB5/p1664292652688569)

To satisfy this issue, we only need to do: 

get_flight_info_statement("select 'hello, world!";)
avantgardnerio commented 1 year ago

It appears this is a good starting point: https://github.com/wangfenjin/arrow-rs/commit/7da031d7debef6ef683e28bc0961b63227dc4a5f

avantgardnerio commented 1 year ago

@DhamoPS should we close this issue as a duplicate of https://github.com/apache/arrow-rs/issues/1413 ?