cloudyr / aws.s3

Amazon Simple Storage Service (S3) API Client
https://cloud.r-project.org/package=aws.s3
381 stars 147 forks source link

put_bucket failing with InvalidLocationConstraint #171

Closed DavidRGriswold closed 6 years ago

DavidRGriswold commented 6 years ago

When using the us-east-1 region, put_bucket is failing with an "Invalid Location constraint" error. It seems the location-constraint header should be left blank when using the us-east-1 region, according to the api documentation here: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

This error is occurring on the latest version, 0.3.8

DavidRGriswold commented 6 years ago

I tried removing the tags put_bucket function and received a MalformedXML error; I am out of my depth here, I'm afraid. Solved my own problem by simply creating the bucket using the CLI, but this should probably be fixed by somebody who knows what they are doing.

leeper commented 6 years ago

Can you give a reproducible example that generates this error? And include the output of sessionInfo() in your example?

DavidRGriswold commented 6 years ago

Sure. I just tried put_bucket("test-bucket",region="us-east-1",acl="public-read")

and got the following output (I've removed some of the long char strings because I don't know what might be auth code stuff)

List of 5
 $ Code              : chr "InvalidLocationConstraint"
 $ Message           : chr "The specified location-constraint is not valid"
 $ LocationConstraint: chr "us-east-1"
 $ RequestId         : chr "653029B86A9C4718"
 $ HostId            : chr "--something long--"
 - attr(*, "headers")=List of 7
  ..$ x-amz-request-id : chr "653029B86A9C4718"
  ..$ x-amz-id-2       : chr "--something long--"
  ..$ content-type     : chr "application/xml"
  ..$ transfer-encoding: chr "chunked"
  ..$ date             : chr "Wed, 11 Oct 2017 19:43:04 GMT"
  ..$ connection       : chr "close"
  ..$ server           : chr "AmazonS3"
  ..- attr(*, "class")= chr [1:2] "insensitive" "list"
 - attr(*, "class")= chr "aws_error"
 - attr(*, "request_canonical")= chr "PUT\n/test-bucket/\n\nhost:s3.amazonaws.com\nx-amz-date:20171011T194305Z\n\nhost;x-amz-date\--something long--"| __truncated__
 - attr(*, "request_string_to_sign")= chr "AWS4-HMAC-SHA256\n20171011T194305Z\n20171011/us-east-1/s3/aws4_request\--something long--"| __truncated__
 - attr(*, "request_signature")= chr "AWS4-HMAC-SHA256 Credential=A--something long--/20171011/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-d"| __truncated__
NULL
Error in parse_aws_s3_response(r, Sig, verbose = verbose) : 
  Bad Request (HTTP 400).

my sessionInfo() output is

R version 3.4.2 (2017-09-28)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.3 LTS

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] aws.s3_0.3.8      devEMF_3.6        shinyjs_0.9.1     pander_0.6.1      rmarkdown_1.6     cowplot_0.8.0     ggplot2_2.2.1     jsonlite_1.5     
 [9] googleAuthR_0.5.1 shiny_1.0.5      

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.12        compiler_3.4.2      plyr_1.8.4          base64enc_0.1-3     tools_3.4.2         testthat_1.0.2      digest_0.6.12      
 [8] memoise_1.1.0       evaluate_0.10.1     tibble_1.3.4        gtable_0.2.0        rlang_0.1.2         rstudioapi_0.7      curl_3.0           
[15] xml2_1.1.1          httr_1.3.1          withr_2.0.0         stringr_1.2.0       knitr_1.17          devtools_1.13.3     rprojroot_1.2      
[22] grid_3.4.2          R6_2.2.2            magrittr_1.5        scales_0.5.0        backports_1.1.1     htmltools_0.3.6     aws.signature_0.3.5
[29] mime_0.5            xtable_1.8-2        colorspace_1.3-2    httpuv_1.3.5        labeling_0.3        miniUI_0.1.1        stringi_1.1.5      
[36] lazyeval_0.2.0      munsell_0.4.3       crayon_1.3.4
leeper commented 6 years ago

Thanks. Yes, the issue is here: https://github.com/cloudyr/aws.s3/blob/master/R/put_bucket.R#L31-L32 The body should only be passed when region != "us-east-1". Will try to fix soon.

leeper commented 6 years ago

This should now be fixed. Sorry for the delay in getting to it.