Add support for S3 object lambda

[ghost]

Is your feature request related to a problem? Please describe. Soto doesn't handle S3 operations for S3 object lambdas.

Describe the solution you'd like With Boto 3 :

import boto3

def lambda_handler(event, context):
    s3 = boto3.client("s3")
    obj = s3.get_object(Bucket="arn:aws:s3-object-lambda:[region]:[account]:accesspoint/[S3 object lambda access point]", Key="object key")
    print(obj["Body"].read())

It works ! The object lambda is successfully fetched and printed.

With Soto, this would ideally work the same way :

(using Soto 6.0.0 9c938aadbbb33d6ed54d04dd6ba494f7f12e0905)

    func testGetObjectFromSoto() async throws {
        let s3 = S3(client: app.aws.client)
        try await s3.getObject(.init(
            bucket: "arn:aws:s3-object-lambda:[region]:[account]:accesspoint/[S3 object lambda access point]",
            key: "object key"
        ))
    }

but it doesn't : SotoSignerV4/signer.swift:313: Fatal error: Unexpectedly found nil while unwrapping an Optional value (The generated URI isn't a valid one and the signature can't be computed).

Describe alternatives you've considered I also tried different ways with a custom endpoint, for example, with an empty bucket parameter :

    func testGetObjectFromSotoCustomEndpoint() async throws {
        let s3 = S3(client: app.aws.client, endpoint: "https://[S3 object lambda access point name]-[account].s3-object-lambda.[region].amazonaws.com")
        try await s3.getObject(.init(
            bucket: "",
            key: "object key"
        ))
    }

It doesn’t work either : <unknown>:0: error: -[... testGetObjectFromSotoCustomEndpoint] : failed: caught error: "The operation couldn’t be completed. (SotoCore.AWSClientError error 1.)" (It seems the object key gets into the hostname in place of the bucket name so it can't possibly work).

I am currently preparing and sending the request with async-http-client and using the Soto's signer manually to generate the authentication header. But a solution within Soto would be preferable.

[adam-fowler]

It is treating your s3 object lambda access point as a bucket name and constructing a URL with this in the host name. I'll have to investigate how these are setup.

Is it possible in boto to set up a verbose state where it prints what endpoints it is hitting? That would give me a hint as to how to proceed. Would you be able to provide this information

[ghost]

I didn't find a configuration for boto where it would be more verbose.

What I can give you is the (approximate) code I wrote in Swift which works.

As you said, the key information is how the endpoint is constructed : it's [object-lambda-access-point-name]-[account-id].s3-object-lambda.[region].amazonaws.com. I assume boto detects that the bucket argument is an ARN, parses it and turns it into this endpoint name format.

Here's the snippet :

func getObject(bucketAccessPoint: String, key: String) async throws -> Data? {
    // Where bucketAccessPoint would be object-lambda-access-point-name-account-id.s3-object-lambda.region.amazonaws.com
    // For example my-object-lambda-access-point-1234567890.s3-object-lambda.eu-west-3.amazonaws.com
    let awsServiceConfig = AWSServiceConfig(
        region: .someregion1,
        partition: .aws,
        service: "s3-object-lambda",
        serviceProtocol: .restxml,
        apiVersion: "2006-03-01"
    )
    let url = "https://\(bucketAccessPoint)/\(key)"
    let signed = try await awsClient.signHeaders(
        url: URL(string: url).unwrap(),
        httpMethod: .GET,
        body: .empty,
        serviceConfig: awsServiceConfig
    )
    let res = try await httpClient.get(URI(string: url), headers: signed)
    return res.body
}