board_s3 serialisation error

[biggiemikkie]

Dear developers! I have faced an issue exploring your package bucket = "dummy_bucket" prefix = "dummy_key" access_key = "*********" secret_key = "**********" board <- board_s3(bucket = bucket, prefix = prefix, access_key = access_key, secret_access_key = secret_key, region = "us-east-1") When I execute the code I get Error: SerializationError (HTTP 403). failed to read from query HTTP response body

Same credentials works well with aws.s3 package for read/write/list operations in AWS S3

My work environment: R 3.6.2, pins 1.0.1, proxy configured with Sys.setenv

Thank you for your contribution in R ecosystem!

[davidpickering]

I too am a regular user of the aws.s3 package and would like to switch to Pins given its features. I am having an issue with this too.

I tried to access an existing bucket:

 board_name = board_s3(
   bucket = "bucket_name", 
   access_key = Sys.getenv("AWS_ACCESS_KEY_ID"),
   secret_access_key = Sys.getenv("AWS_SECRET_ACCESS_KEY"),
   region = Sys.getenv("AWS_DEFAULT_REGION"))

I also tried creating a new bucket

board_s3("pins-test-hadley",
          access_key = Sys.getenv("AWS_ACCESS_KEY_ID"),
          secret_access_key = Sys.getenv("AWS_SECRET_ACCESS_KEY"),
          region = Sys.getenv("AWS_DEFAULT_REGION"))

Troubleshooting:

• I checked my argument names
• I checked my env values
• creating a new bucket

[juliasilge]

I believe that this is about authentication plus maybe whether the bucket exists.

For example, I personally cannot use any of those keys and am required to use AWS SSO for our AWS instance. Here is what I did:

• I configured AWS SSO so that I have a file at ~/.aws/config
• I logged in via aws sso login --profile PowerUser-158990117310
• I used that same profile name to authenticate

library(pins)

b <- board_s3(
  "pins-testing-julia", 
  profile = "PowerUser-158990117310", 
  region = "us-east-2"
)
b %>% pin_write(mtcars, "my-lovely-cars")
#> Guessing `type = 'rds'`
#> Creating new version '20220909T000803Z-66143'
#> Writing to pin 'my-lovely-cars'

^{Created on 2022-09-08 with reprex v2.0.2}

Here I am using a bucket that I created via:

svc <- paws::s3(
  config = list(
    credentials = list(profile = "PowerUser-158990117310"),
    region = "us-east-2"
  )
)

svc$create_bucket(
  Bucket = "pins-testing-julia", 
  CreateBucketConfiguration = list(
    LocationConstraint = "us-east-2"
  ))

I don't believe the current implementation of board_s3() will create a bucket for you; it only checks to see if the bucket is there: https://github.com/rstudio/pins-r/blob/7379792150e6b99cc3704f93e585c085aee67252/R/board_s3.R#L108

🎯 Since you are already using the paws package, can you use the same authentication method that you normally use (that works) and try to write to an existing bucket that you are allowed to write to?

We should address board_s3() not creating buckets, probably via documentation ("you can only write to an existing bucket") or possible by creating the bucket if it doesn't exist.

[juliasilge]

We have updated the documentation for board_s3() in #659 to clarify that you need to have an existing bucket. I think that, combined with general paws authentication support, will get you where you need to go! Let us know if you have further problems. 🙌

[github-actions[bot]]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.