search

moj-analytical-services / s3tools

Interact with files in s3 on the Analytical Platform
8 stars 2 forks source link

quick fix for function #54

Closed ThomasHepworth closed 3 years ago

ThomasHepworth commented 3 years ago

I should note that this fix only works for the latest version of aws.s3 - version 0.3.22. Users running off version 0.3.21 will have s3tools work as expected.

renv users now get the following error when trying to use the original s3tools::write_df_to_csv_in_s3 function Screenshot 2021-08-03 at 17 31 47

The below should resolve the issue. It looks like the issue has arisen due to amazon (or whoever is maintaing aws.s3) changing their aws.s3::put_object function, such that it now checks the file size of our rawConnection object and falls over.

You can recreate this error using:

rc <- rawConnection(raw(0), "r+")
write.csv(data.frame(x = 1:3), rc)
file.size(rc)

I've also added in some quick regex to ensure the user enters a valid filepath. If you simply enter an s3 bucket name, you'll find a filepath of NA/entered_filepath when you browse to your S3 bucket.

ThomasHepworth commented 3 years ago

I'll just be lazy and assign you, Dan - I don't think anyone else is maintaining this at present