How to enforce aws signature to v2?

[charz]

I found some signature types in test file (https://github.com/elastic/elasticsearch-cloud-aws/blob/5390c01b1cb5f8c916050e3b1776aa86a91f9d1b/src/test/java/org/elasticsearch/cloud/aws/AWSSignersTest.java).

QueryStringSignerType
AWS3SignerType
AWS4SignerType
NoOpSignerType

My question is how to set signature to V2?

[dadoonet]

I guess you can not.

[conzetti]

@charz We also run a S3-compatible service that requires http_aws2 (V2), and I found in testing (and after reading others posts about Ceph interoperability), that S3SignerType could be used. However, after testing this, I've been unsuccessful at getting this to function on my ES 1.5.2 clusters. It does, however, work on ES 1.6.0. This will log errors, though, such as:

wrong signer set for [cloud.aws.s3.signer] or [cloud.aws.signer]: [S3SignerType]

You can see the logic leveraged to prevent unwanted signers being called here.

Please let us know your results. I've been bumping into an issue that prevents the S3SignerType from being used, and the plugin defaults to http_aws4.

[dadoonet]

It should be AWS3SignerType, right?

[conzetti]

@dadoonet, I assumed so, based on your code, documentation, and the aws sdk. Then when I saw issue 155 opened by @armin-bauer, and @herviou's comments about leveraging S3SignerType to eliminate the regional configuration (which I'm not using), I gave it a shot.

I discovered on ES 1.5.2, I can actually PUT objects, but other operations fail:

Operation	Result
DELETE	operation will attempt to use http_aws4 (AWS4SignerType)
GET	operation will attempt to use http_aws4 (AWS4SignerType)
HEAD	operation will attempt to use http_aws4 (AWS4SignerType)
PUT	operation will attempt to use http_aws2 (S3SignerType)

So, when leveraging something like curator to perform the snapshot offloading to our object store, the process fails initially due to an inventory of snapshots that's run (queries [GET] /_snapshot/repo/_all). On our side, we throw a HTTP/403, because the signing method isn't what we're expecting. I'm still trying to find the RC, and will PR against your code if I figure it out.

@charz: apologies for hijacking the thread.

[charz]

@conzetti That's fine. I think we have same issue. When I trigger snapshot with S3SignerType or AWS3SignerType, PUTs look good (operating with http_aws2), but GETs use http_aws4. That causes the snapshot failure, but you can see some data already upload to server.

[jamshid]

Found while issue while searching for a solution to make aws-sdk-java work with an S3-compatible service that does not implement V4 signatures. To force aws-sdk-java to use V2 you have to use a recent SDK release and configure setSignerOverride("S3SignerType"). Without this, the sdk always uses V4 signatures for GetObjet.

        ClientConfiguration opts = new ClientConfiguration();
        opts.setSignerOverride("S3SignerType");  // NOT "AWS3SignerType"
        AmazonS3Client s3 = new AmazonS3Client(opts);
        s3.setEndpoint("https://storage.example.com");

See https://github.com/aws/aws-sdk-java/issues/372.

[dadoonet]

@jamshid Thank you. I think it makes sense. So I basically need to change this https://github.com/elastic/elasticsearch-cloud-aws/blob/master/src/main/java/org/elasticsearch/cloud/aws/AwsSigner.java#L44-51 and the doc?

[xuzha]

Close this via https://github.com/elastic/elasticsearch/pull/13360