search

apache / arrow-adbc

Database connectivity API standard and libraries for Apache Arrow
https://arrow.apache.org/adbc/
Apache License 2.0
362 stars 89 forks source link

[Java] Question about new driver #1822

Open HaoXuAI opened 5 months ago

HaoXuAI commented 5 months ago

What would you like help with?

I'm thinking of implementing an adbc driver for AWS Athena. And our codebase is in java. It looks like there is no driver in java implemented yet. So is it more preferred to implement it in other language like C++ or Go and use it in Java?

lidavidm commented 5 months ago

For the time being, Java can only use drivers written in Java unfortunately.

(Also, it appears Athena is row-oriented, so I'm not sure you could expect much speedup?)

lidavidm commented 5 months ago

That said, a driver would be welcome, whether in Java or not (I still would like to add the JNI bridge but just don't have time)

HaoXuAI commented 5 months ago

Athena can use parquet as storage format, so it should work with arrow column format? I'll look deeper into it see if that could work. I'm not familiar with JNI, but intested in learning it as well. :)

jduo commented 4 months ago

There is a JDBC driver for Athena: https://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html So it'd be feasible to use the Java version of ADBC + the ADBC driver for JDBC sources to connect to it.