noctua: data not shown in s3 if overwrite=TRUE, but shown if overwrite=FALSE and append=TRUE

[DyfanJones]

Hi Dyfan,

I am uploading data into s3 buckets and using for that purpose your package “noctua” -- which is great. But, there seems to be one thing I have come across which I have not yet managed after endless testing using this code snippet:

### Connect to database

con <- dbConnect(noctua::athena()

             , aws_access_key_id = aws_access_key_id

             , aws_secret_access_key = aws_secret_access_key

             , aws_session_token = aws_session_token

             , s3_staging_dir = 's3://bucket/path/to/folder/'

             , region = 'us-east-1'

             , schema = 'schema'

             , work_group = 'abc_ef_gh_jklmnoqr'

             )

### Upload to Athena

dbWriteTable(conn = con

           , name = schema.aaa_bbb_cccccc'

           , value = AbcDt

           , overwrite = ifelse(r==1, TRUE, FALSE)

           , append = ifelse(r==1, FALSE, TRUE)

           , file.type = 'json'

           , partition=c("ZEITSTEMPEL" = format(Sys.Date(), "%Y%m%d"))

           , s3.location = 's3://bucket/path/to/folder/'

           )

I am using this code in a loop. By doing so, I have noticed that for the first loop (r==1), the data can be queried in Athena but does not seem to be available or visible in s3. For the following loops (r>1), i.e. if the data is appended, the data is shown in s3, too.

Is this anything you can reproduce?

Many thanks in advance (and for the great work you have put into the package) Christian

[DyfanJones]

This issue has been fixed through emails