object PUT operations require `Content-Length` in the header

[almartin82]

per the doc Content-Length is a required header for a PUT operation on a bucket. @cboettig perhaps this helps with #5? Not sure if it also applies to POST operations (haven't delved into that doc yet)

[leeper]

This appears to work, but I've only tested it with one file.

[almartin82]

I was able to get it to PUT a folder into my bucket, but not a file. Here's my initial work on getting content-length and the object content to pass to aws.signature

[leeper]

I just updated aws.signature so that we don't have to read the whole file into memory to get its hash, which will be hugely important for large files. You'll need to reinstall it. cc @cboettig

So then I just changed this package back to work with that flow. I was able to upload and retrieve an object using the current master branch. What was the code you were using?

[almartin82]

oh, nice - just saw the upload_file edits to s3HTTP. I was thinking about how to do that on the way in to work. Now I don't have to :+1: :+1:

[almartin82]

@leeper the version on my branch -- not your stuff!

[leeper]

Side note: I'd prefer not use magrittr anywhere in the package, as it's not necessary for anything we're doing here. Just a personal preference. Hope that doesn't bother you.

[almartin82]

yep, totally. not a problem! just a pipe operator fan.

[almartin82]

definitely getting closer -- I tried locally and got a 400, not a 404

$Code
[1] "XAmzContentSHA256Mismatch"

$Message
[1] "The provided 'x-amz-content-sha256' header does not match what was computed."

$ClientComputedContentSHA256
[1] "e9951059b854266a38c1588286a2d3b6873ea64fc04db1241c2a6dd147bb09f4"

$S3ComputedContentSHA256
[1] "326f82b4d040693cf798be42a4e0b86c335a0ab88b6d871ae463ca88c10599c9"

[almartin82]

@leeper here is (hopefully) a reproducible example:

test_that("basic usage of putobject for signed in user", {
  fileConn <- file("s3test.txt")
  writeLines(c("cloudyr", "test"), fileConn)
  close(fileConn)

  ex <- putobject(
    file = 's3test.txt',
    bucket = 'hpk', 
    object = 's3test.txt',
    key = Sys.getenv("TRAVIS_AWS_ACCESS_KEY_ID"), 
    secret = Sys.getenv("TRAVIS_AWS_SECRET_ACCESS_KEY")
  )

  if (file.exists('s3test.txt')) file.remove('s3test.txt')
})

returns

$Code
[1] "XAmzContentSHA256Mismatch"

$Message
[1] "The provided 'x-amz-content-sha256' header does not match what was computed."

$ClientComputedContentSHA256
[1] "2a9f2e2f3792576c5bf6155a3a83a7834fd5832d177aaf1f2f580ec5cd339c67"

$S3ComputedContentSHA256
[1] "d3b0a764aeba588d3dd98a67cd896a6ca3732ea2122d11f4715d26fd32ff2f22"

$RequestId
[1] "BD0EC478E138F5E3"

$HostId
[1] "Qlfb/3gxUK5S7ZDcvbxt0nEMJKCQDfolpzxe7wVkURi8HZZT/DYq/fZUPoNEjS8X"

attr(,"headers")
$`x-amz-request-id`
[1] "BD0EC478E138F5E3"

$`x-amz-id-2`
[1] "Qlfb/3gxUK5S7ZDcvbxt0nEMJKCQDfolpzxe7wVkURi8HZZT/DYq/fZUPoNEjS8X"

$`content-type`
[1] "application/xml"

$`transfer-encoding`
[1] "chunked"

$date
[1] "Sat, 25 Jul 2015 02:26:14 GMT"

$connection
[1] "close"

$server
[1] "AmazonS3"

attr(,"class")
[1] "insensitive" "list"       
attr(,"class")
[1] "aws_error"

[leeper]

Hmm...I can get this all to work using my own keys, but it doesn't seem to work using the keys you shared. Do they have limited permissions in some way?

[almartin82]

success! forgot to force the update for aws.signature. working for both my keys and the travis keys, locally.

[leeper]

Okay, great!