search

cloudyr / aws.s3

Amazon Simple Storage Service (S3) API Client
https://cloud.r-project.org/package=aws.s3
381 stars 147 forks source link

Support Athena queries #248

Open dylancis opened 6 years ago

dylancis commented 6 years ago

Hi there- Python boto3 equivalent package no supports Athena; boto3-athena I would be great if aws.s3 could also supports Amazon Athena - at the moment the only way to run Athena via R is to use the JDBC driver and having rJava installed.

Thank you, Dykan

leeper commented 6 years ago

Yep, it's in the pipeline. Forthcoming over here: https://github.com/cloudyr/aws.athena

RobinL commented 6 years ago

Worth noting there are big differences between the performance of the various Athena drivers. Specifically, with some of the drivers, the speed at which query results are received by R is slow. So if you do a select * from a big table, it can be 10s of minutes to return the data, even if the query itself ran very fast.

The latest JDBC driver is much faster.

There also seem to be differences in handling of data types between the drivers.