Closed DyfanJones closed 2 years ago
Implementation seems to be working nicely :D
library(DBI)
library(noctua)
library(dplyr, warn.conflict = FALSE)
con <- dbConnect(athena())
noctua_options(cache = 10, unload = TRUE)
df = dbGetQuery(con, "select * from awswrangler_test.noaa")
noaa = tbl(con, dbplyr::in_schema("awswrangler_test", "noaa"))
noaa %>% filter(!is.na(m_flag))
noctua:::athena_option_env$cache_dt
#> QueryId Query State StatementType WorkGroup UnloadDir
#> 1: 774b6a83-cf53-4a5b-849c-d4c144bcf672 SELECT *\nFROM "awswrangler_test"."noaa"\nWHERE (NOT((("m_flag") IS NULL)))\nLIMIT 11 SUCCEEDED DML primary 08313f36-6da9-490f-8ba0-947f14d9a9f8
#> 2: 9a94ae61-8b85-4620-9440-9033990dc83f select * from awswrangler_test.noaa SUCCEEDED DML primary 72c55e13-6770-4b15-8e98-bc77be1e5c8b
Currently
noctua
supportsAWS Athena unload
method through adhoc setting of parameterunload
indbGetQuery
,dbSendQuery
anddbExecute
.Unfortunetly this means that
dplyr
functionality doesn't benefit from this feature. By having a method to setunload = TRUE
on a package level thendplyr
can take advantage of this feature with little implementation ondplyr
side of things.