Using dbGetQuery() with unload = TRUE returns a 0-row data frame when s3_staging_dir is set to a bucket with no key prefix e.g. s3://temp-bucket instead of s3://temp-bucket/results. This is due to a couple separate issues:
1.
The helper function split_s3_uri returns the bucket name when the s3_staging_dir is a bucket only and has no trailing slash. E.g.:
Issue
Using
dbGetQuery()
withunload = TRUE
returns a 0-row data frame whens3_staging_dir
is set to a bucket with no key prefix e.g.s3://temp-bucket
instead ofs3://temp-bucket/results
. This is due to a couple separate issues:1.
The helper function
split_s3_uri
returns the bucket name when thes3_staging_dir
is a bucket only and has no trailing slash. E.g.:This results in the following
key
value construction in the.fetch_unload
helper:Which is passed to
list_objects_v2
and results in anull
set of keys, even if they actually exist.2.
Similarly, the
split_s3_uri
helper returns the following when the staging directory has a trailing slash:Which results in the following
bucket
andkey
combo inside.fetch_unload
:The prefixed forward slash in the key again results in null results when passed to
list_objects_v2
inside.fetch_unload
.