InvalidLocationConstraint while registering snapshot repo in s3

[khinlatt]

This is a similar problem to https://github.com/elastic/elasticsearch-cloud-aws/issues/73

I'm running es-1.7.3 and cloud-aws-2.7.1

When I run this to register snapshot to s3,

curl -XPUT 'http://localhost:9200/_snapshot/my_s3_repository' -d '{
    "type": "s3",
    "settings": {
        "bucket": "snapshots.example",
        "region": "us-east-1"
    }
}'

I get this error.

{"error":"RepositoryException[[my_s3_repository] failed to create repository]; nested: CreationException[Guice creation errors:\n\n

1) Error injecting constructor, com.amazonaws.services.s3.model.AmazonS3Exception: The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint; Request ID: XXXXX), S3 Extended Request ID: XXXXX  at org.elasticsearch.repositories.s3.S3Repository.<init>(Unknown Source)\n  while locating org.elasticsearch.repositories.s3.S3Repository\n  while locating org.elasticsearch.repositories.Repository\n\n1 error];

 nested: AmazonS3Exception[The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint; Request ID: XXXXX)]; ","status":500}

Anybody has any idea on why this error is shown? Thanks in advance!

[khinlatt]

Update: the error was because I have not created that bucket on s3.

New error: now it fails with Access Denied

{"error":"RepositoryVerificationException[[my_s3_repository] path  is not accessible on master node]; nested: IOException[Unable to upload object tests-dfjo2894g9d9uvxgivhez-master due to AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: XXXXXX)]; ","status":500}

I do have the config set up in elasticsearch.yml like so:

cloud.aws.access_key: XXX
cloud.aws.secret_key: XXX

I have policy set up like this:

{
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucketVersions"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::my_bucket_name"
            ]
        },
        {
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::my_bucket_name/*"
            ]
        }
    ],
    "Version": "2012-10-17"
}

[dadoonet]

@khinlatt Thanks for closing the issue. Out of curiosity, what was the final issue? I mean how you fixed it?

[khinlatt]

@dadoonet the policy from this documentation did not work for me. The moment I changed it to s3 full access, it was good to go.

Has anyone have that issued with the policy before?

[dadoonet]

Not that I'm aware of. But good to know. Thanks

[khinlatt]

@dadoonet after I have set this process up successfully on my local machine, I am doing the same on an ec2 box. With proper creds set up, including even s3 full access. I am getting this error: Running from ec2 box:

curl -XPUT 'http://localhost:9200/_snapshot/es_snapshot' -d '{
"type": "s3",
"settings": {
"bucket": "mybucket",
"region": "us-east-1"
}
}'

{"error":"RepositoryVerificationException[[es_snapshot] path  is not accessible on master node]; nested: IOException[Unable to upload object tests-dcqKHPk4TNSDFjo5sutL0uw-master due to AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: B4fEE4FC9642F42D)]; ","status":500}

Sorry to bug you again. Could you please shed some light on this issue? I am running es1.7 and elasticsearch-cloud-aws2.7.1. Thanks a lot.

Updates: Seems like cloud.aws.access_key and cloud.aws.secret_key is not being taken in although I restarted ES multiple times. When I specify the access keys in the curl command like {"settings": {"bucket": "", "region": "", "secret_key": "", "access_key": ""}}, it works.

[flybd5]

This whole business of registering S3 buckets appears to be totally screwed up at AWS. People have been reporting problems since May and they're not responding to the issues on a timely basis.

[jeffb-stell]

This might be due to the API not accepting us-east-1 for a region: https://github.com/boto/boto3/issues/125 (there was also an issue opened for the javascript aws sdk)