bippio / go-impala

Golang Driver for Apache Impala
MIT License
51 stars 40 forks source link

Design Change - Move to HS2 interface to Impala rather than beeswax #6

Closed vishjosh closed 5 years ago

vishjosh commented 5 years ago

Suggestion from @timarmstrong . @intamyuto - let us discuss if this is something we can start working on in parallel . We can get a lot of help from the community and Tim has already pointed us to a good set of resources . His C++ client seems to be very relevant to our go driver.

Suggestion Details

I'd strongly suggest to use the HS2 interface to Impala (port 21050 by default) rather than beeswax. Beeswax is a legacy thing that's really only kept alive because impala-shell hasn't been ported to use HS2. I've been starting to advocate for doing that work and removing beeswax from Impala in the future. Generally HS2 makes a lot more sense, is exercised by more different clients, and should be easier to work with. E.g. the text-based beeswax protocol can't handle tabs properly - https://github.com/bippio/go-impala/blob/master/rowset.go#L171 and you don't need to do string parsing in the client - https://github.com/bippio/go-impala/blob/master/rowset.go#L203. I think the hard stuff like SSL/LDAP should all be the same.

Here's the thrift interface - https://github.com/apache/impala/blob/master/common/thrift/hive-1-api/TCLIService.thrift#L1138. You get back the data as a set of columns in the native types - https://github.com/apache/impala/blob/master/common/thrift/hive-1-api/TCLIService.thrift#L395. Impyla and the C++ hs2client (not finished, but quite clean code) both use that interface - https://github.com/cloudera/impyla, https://github.com/cloudera/hs2client, along with the standard JDBC and ODBC clients.

vishjosh commented 5 years ago

@paribhasha Please add your comments on making the driver consistent with Golang's sql/driver way of doing stuff

shaloi commented 5 years ago

A redesign of the client driver should be done in conformance with Golang's database/sql by implementing the interface:

https://golang.org/pkg/database/sql/ https://github.com/golang/go/wiki/SQLInterface

shaloi commented 5 years ago

Resolved via PR#12