myENA / consul-backinator

Command line Consul backup and restore utility supporting KVs, ACLs and Queries
Mozilla Public License 2.0
226 stars 22 forks source link

Failed to backup key data: InvalidBucketName #30

Closed nani-ar closed 7 years ago

nani-ar commented 7 years ago

did docker run myena/consul-backinator backup -addr=10.0.1.50:8500 -file=s3://"access key id":"secret access key"@s3-bucket name/consul_backup?region=us-west-2

i am getting failed to backup data : invalid bucket name : the specified bucket is not valid,

but my nucket name is correct, can someone help ?

aaronhurt commented 7 years ago

The URI shouldn't contain spaces. That includes the access key, secret and bucket name. You also shouldn't need to quote the key/secret. The standard access keys and secrets should not contain spaces and should be URL safe.

nani-ar commented 7 years ago

i did docker run myena/consul-backinator backup -addr=10.0.1.50:8500 -file=s3://access key id:secretaccesskey@s3-bucket name/consul_backup?region=us-west-2

I am still getting invalid bucket name.

aaronhurt commented 7 years ago

Are you using keys that have access to that bucket? Can you access/write to that bucket using a standard S3 client and the same keys?

nani-ar commented 7 years ago

yes, "aws s3 cp" works fine. The error message was 400 bad request.

docker run myena/consul-backinator backup -addr=10.0.1.50:8500 -file=s3://access key id:secretaccesskey@anilmytest/Backup?region=us-west-2 2017/08/30 03:48:45 [Error] Failed to backup key data: InvalidBucketName: The specified bucket is not valid. status code: 400, request id: 07FF4C8642197674, host id: qMWisB4qNbA6WqZZ8916Uhl95Mpkb2M14X/JWyU5N4O3VtC3jwvnpC6a4x5VyIt33MfC4HXi+Pg=

aws s3 cp test_file s3://anilmytest/Backup

upload: ./test_file to s3://anilmytest/Backup [#

aaronhurt commented 7 years ago

@nanidvps Going to try and reproduce this today. What version of the Docker image are you running?

aaronhurt commented 7 years ago

@nanidvps Okay, I was able to reproduce this. There appears to be a bug in the URI parsing. You can workaround for the moment by passing them as environment variables instead of in the URI ... example:

docker run -e AWS_ACCESS_KEY_ID=key -e AWS_SECRET_ACCESS_KEY=secret myena/consul-backinator backup -addr=cnl01.dev.ena.net:8500 -file=s3://ahurt-testing/junk/backup?region=us-east-1

I'll try and get the parsing fixed today.

aaronhurt commented 7 years ago

@nanidvps Please try the latest tag after the automated build finishes and let me know if this works.

nani-ar commented 7 years ago

Status: Downloaded newer image for myena/consul-backinator:latest docker run myena/consul-backinator backup -addr=10.0.1.50:8500 -file=s3://access key id:secretaccesskey@anilmytest/Backup?region=us-west-2 [Error] Failed to backup key data: BucketAlreadyOwnedByYou: Your previous request to create the named bucket succeeded and you already own it. status code: 409, request id: 3F231DD26FB7B637, host id: sQDegGUjvnzlT5YH7lcwZzLHmZPd1gNUBLKrB65VPKgiifX1rImHJPexGjwa/kT41/Xi6RTLlv8=

aaronhurt commented 7 years ago

That's a new one. What do your ACLs look like for the key you are testing? I haven't seen that with my ACLs.

aaronhurt commented 7 years ago

Looks like this is affecting other go projects as well.

https://github.com/terraform-providers/terraform-provider-aws/issues/423

I'll do a bit more investigating.

aaronhurt commented 7 years ago

Looks like it's documented in the newer SDK package versions:

https://github.com/aws/aws-sdk-go/blob/master/service/s3/api.go#L296

nani-ar commented 7 years ago

working with the latest build, thanks

aaronhurt commented 7 years ago

Thank you!