search

DyfanJones / noctua

Connect R to Athena using paws SDK (DBI Interface)
https://dyfanjones.github.io/noctua/
Other
45 stars 5 forks source link

dbWriteTable doesn't seem to be fully repairing the table #81

Closed DyfanJones closed 4 years ago

DyfanJones commented 4 years ago

A user on Rstudio community is experiencing issues were dbWriteTable isn't repairing the table once data has been updated to AWS Athena link.

This could be down to dbExecute not fully reporting errors from AWS Athena and not notifying user the repair has failed.

Might need to explore using ALTER TABLE instead of MSCK REPAIR for general speed if user has multiple partition.

Note: dbExecute has been fixed in latest PR #80 and will return AWS Athena errors. Will need to check what version user is using and then debug

DyfanJones commented 4 years ago

Closing Ticket as MSCK REPAIR was performed on AWS Athena, however user was using HIVE and Hive Metadata Store wasn't updated. As noctua doesn't connect to Hive, user has been given advice in how to tackle issue going forward.

Note: re-open if solution for user doesn't resolve issue