krobertson / deb-s3

Easily create and manage an APT repository on S3 -- NO LONGER MAINTAINED
MIT License
482 stars 148 forks source link

Doesn't work with new S3 locations #114

Closed thusoy closed 7 years ago

thusoy commented 7 years ago

Using the exact same IAM policies and identical deb-s3 commands (apart from the target bucket), uploading to US default works while Frankfurt fails with a BadRequest:

>> Retrieving existing manifests
/var/lib/gems/2.1.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:375:in `return_or_raise': AWS::S3::Errors::BadRequest (AWS::S3::Errors::BadRequest)
    from /var/lib/gems/2.1.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:476:in `client_request'
    from (eval):3:in `head_object'
    from /var/lib/gems/2.1.0/gems/aws-sdk-v1-1.66.0/lib/aws/s3/s3_object.rb:296:in `head'
    from /var/lib/gems/2.1.0/gems/aws-sdk-v1-1.66.0/lib/aws/s3/s3_object.rb:273:in `exists?'
    from /var/lib/gems/2.1.0/gems/deb-s3-0.8.0/lib/deb/s3/utils.rb:59:in `s3_exists?'
    from /var/lib/gems/2.1.0/gems/deb-s3-0.8.0/lib/deb/s3/utils.rb:63:in `s3_read'
    from /var/lib/gems/2.1.0/gems/deb-s3-0.8.0/lib/deb/s3/release.rb:30:in `retrieve'
    from /var/lib/gems/2.1.0/gems/deb-s3-0.8.0/lib/deb/s3/cli.rb:169:in `upload'
    from /var/lib/gems/2.1.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
    from /var/lib/gems/2.1.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
    from /var/lib/gems/2.1.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
    from /var/lib/gems/2.1.0/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
    from /var/lib/gems/2.1.0/gems/deb-s3-0.8.0/bin/deb-s3:8:in `<top (required)>'
    from /usr/local/bin/deb-s3:22:in `load'
    from /usr/local/bin/deb-s3:22:in `<main>'

I'm guessing this is due to the new authentication method on the newer S3 locations, and that this would be fixed by #80, but I'm not sure.

krobertson commented 7 years ago

Yes, I believe you're correct.

I had a WIP branch with the newer gem, I'll take a look at what I had left to finish it off.

pedrohdz commented 7 years ago

Not sure if this is a related issue, or something different. First time deb-s3 user here so I could be messing something up.

I'm trying to create a repo for the first time with deb-s3 upload and I keep getting the following:

>> Retrieving existing manifests
/var/lib/gems/2.3.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:375:in `return_or_raise': AWS::Errors::Base (AWS::Errors::Base)
        from /var/lib/gems/2.3.0/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:476:in `client_request'
        from (eval):3:in `head_object'
        from /var/lib/gems/2.3.0/gems/aws-sdk-v1-1.66.0/lib/aws/s3/s3_object.rb:296:in `head'
        from /var/lib/gems/2.3.0/gems/aws-sdk-v1-1.66.0/lib/aws/s3/s3_object.rb:273:in `exists?'
        from /var/lib/gems/2.3.0/gems/deb-s3-0.8.0/lib/deb/s3/utils.rb:59:in `s3_exists?'
        from /var/lib/gems/2.3.0/gems/deb-s3-0.8.0/lib/deb/s3/utils.rb:63:in `s3_read'
        from /var/lib/gems/2.3.0/gems/deb-s3-0.8.0/lib/deb/s3/release.rb:30:in `retrieve'
        from /var/lib/gems/2.3.0/gems/deb-s3-0.8.0/lib/deb/s3/cli.rb:169:in `upload'
        from /usr/lib/ruby/vendor_ruby/thor/command.rb:27:in `run'
        from /usr/lib/ruby/vendor_ruby/thor/invocation.rb:126:in `invoke_command'
        from /usr/lib/ruby/vendor_ruby/thor.rb:359:in `dispatch'
        from /usr/lib/ruby/vendor_ruby/thor/base.rb:440:in `start'
        from /var/lib/gems/2.3.0/gems/deb-s3-0.8.0/bin/deb-s3:8:in `<top (required)>'
        from /usr/local/bin/deb-s3:23:in `load'
        from /usr/local/bin/deb-s3:23:in `<main>'

I created the bucket, in us-west-2, recently.

Is this related? Am I goofing something else up?

Thanks in advance!

manicminer commented 7 years ago

@krobertson do you think you'll be able to release an update that works with newer S3 locations? thanks!

ruland commented 7 years ago

@manicminer Just did a pull request with the new aws sdk, that should make it work for you. you can check out my fork and use it.