KeyTooLongError after multiple S3 requests. Ruby 2.0 only

Meekohi commented 11 years ago

Works fine with ruby1.8.7 but fails with ruby2.0.0. All the jpegs downloads complete, then there is an error when downloading info.json.

ruby-2.0.0-p0/gems/aws-2.9.1/lib/awsbase/awsbase.rb:542:in 'block (2 levels) in request_info_impl': KeyTooLongError: Your key is too long (Aws::AwsError)

def download_file( source_path, s3_key )
  File.open(source_path,'w') {|f|
    @bucket.get(s3_key) do |chunk|
      f.write(chunk)
    end
  }
end

jpegs = @bucket.keys({:prefix => id+"/jpegs"})
jpegs.each do |key|
    download_file("tmp.jpg",key)
end

infojson = @bucket.keys({:prefix => id+"/info.json"})
download_file("info.json",infojson)

Meekohi commented 10 years ago

Still having this issue on a fresh install. ruby 2.0.0p247 (2013-06-27 revision 41674) [universal.x86_64-darwin13]

treeder commented 10 years ago

Could you print all the keys and find out which one it's erroring on?

Meekohi commented 10 years ago

The error occurs on the final download_file("info.json",infojson)

Seems like it might be a case of expanding html_entities over and over again?

REQUEST=mybucket.s3.amazonaws.com:80/%5B%23%3CAws%3A%3AS3%3A%3A... (don't want to show the whole thing since I think the API keys are in there)

appoxy / aws

KeyTooLongError after multiple S3 requests. Ruby 2.0 only #128