DyfanJones / RAthena

Connect R to Athena using Boto3 SDK (DBI Interface)
https://dyfanjones.github.io/RAthena/
Other
35 stars 6 forks source link

Selecting only a datetime column does not work #52

Closed OssiLehtinen closed 4 years ago

OssiLehtinen commented 4 years ago

Issue Description

If a single datetime column is selected, an error is thrown. Didn't have time to dig deeper, but here it is fyi.

Reproducible Example

Example:

library(RAthena)
library(DBI)
con <- dbConnect(RAthena::athena(),
                 profile_name = "rathena")

df <- data.frame(time_col = Sys.time(), other_col = 1:5)
dbWriteTable(con, name="temp.test6", df)
tbl(con, in_schema("temp", "test6")) %>% select(time_col)

Error in setnames(ans, col.names) : Can't assign 1 names to a 2 column data.table In addition: Warning messages: 1: In data.table::fread(File, col.names = names(Type2), colClasses = unname(Type2), : Found and resolved improper quoting in first 100 rows. If the fields are not quoted (e.g. field separator does not appear within any field), try quote="" to avoid this warning. 2: In data.table::fread(File, col.names = names(Type2), colClasses = unname(Type2), :

Error in setnames(ans, col.names) : Can't assign 1 names to a 2 column data.table

DyfanJones commented 4 years ago

Hi @OssiLehtinen, thanks for bringing this to my attention. I believe i have found the issue. The problem is sep = "auto". This is a really useful feature of data.table however a problem when it comes to fields that are return in this format:

2019-12-16 09:09:45.330

It is smart enough to use " " as the separating value. This sadly is not what we want and it is being too smart. As AWS Athena returns csv file types. The separating value will be set to ","

DyfanJones commented 4 years ago

Closing this ticket as PR #51 fixes this issue.