Field DeleteMarker in HeadObjectOutput is not usable

clumio-nick commented 4 months ago

Describe the bug

The documentation for HeadObjectOutput says that the DeleteMarker field is true if the object is a delete marker.

// Specifies whether the object retrieved was (true) or was not (false) a Delete
// Marker. If false, this response header does not appear in the response. This
// functionality is not supported for directory buckets.
DeleteMarker *bool

However, HeadObject always returns an error for delete markers, and it returns a nil HeadObjectOutput when there is an error. This makes it impossible to use the DeleteMarker field to check whether HeadObject failed because the version is a delete marker.

This behavior is different than the behavior of aws-sdk-go, which returns both the output struct and the error when using HeadObject on a delete marker. Using aws-sdk-go, it is possible to check the fields of HeadObjectOutput even when there is an error.

Expected Behavior

I expect to be able to check the DeleteMarker field when using HeadObject on a delete marker.

Current Behavior

HeadObjectOutput is nil when using HeadObject on a delete marker.

Example output from the program below:

HeadObject output: <nil> HeadObject error: operation error S3: HeadObject, https response error StatusCode: 405, RequestID: <...snip...>, HostID: <...snip...>, api error MethodNotAllowed: Method Not Allowed

Reproduction Steps

Run this program to test HeadObject on a delete marker: go run main.go -bucket <bucket> -key <key> -version <version>

package main

import (
    "context"
    "fmt"
    "flag"
    "os"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
    var bucket, key, version string

    flag.StringVar(&bucket, "bucket", "", "bucket containing the delete marker (required)")
    flag.StringVar(&key, "key", "", "delete marker key (required)")
    flag.StringVar(&version, "version", "", "delete marker version ID (required)")
    flag.Parse()

    ctx := context.Background()

    config, err := config.LoadDefaultConfig(ctx)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to load config: %v\n", err)
        os.Exit(1)
    }

    svc := s3.NewFromConfig(config)

    input := &s3.HeadObjectInput{
        Bucket:    aws.String(bucket),
        Key:       aws.String(key),
        VersionId: aws.String(version),
    }

    output, err := svc.HeadObject(ctx, input)

    fmt.Printf("HeadObject output: %v\n", output)
    fmt.Printf("HeadObject error: %v\n", err)
}

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

github.com/aws/aws-sdk-go-v2 v1.25.2 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 github.com/aws/aws-sdk-go-v2/config v1.26.4 github.com/aws/aws-sdk-go-v2/credentials v1.16.15 github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11 github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.2 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.2 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.2 github.com/aws/aws-sdk-go-v2/internal/ini v1.7.2 github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.10 github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.1 github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.10 github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.2 github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.10 github.com/aws/aws-sdk-go-v2/service/s3 v1.48.0 github.com/aws/aws-sdk-go-v2/service/sso v1.18.6 github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.7 github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.7 github.com/aws/aws-sdk-go-v2/service/sts v1.26.7

Compiler and Version used

go version go1.22.0 darwin/amd64

Operating System and version

macOS Sonoma 14.3

lucix-aws commented 3 months ago

@clumio-nick --

This is unfortunately a deficiency in the S3 API as written. From an SDK and AWS protocol perspective, success and error responses are mutually exclusive. The S3 API models this response parameter as a header-bound component of the success response, but not the error response, thus the SDK does not know to provide this information as part of the returned error in that path.

Go v1 is actually the exception here, rather than the rule. Because of the generic way it handles operation return values, you always get a non-nil response, and thus you can technically still see any success response-bound values there. That was not intentional in v1 per se but it is established. The Go v2 behavior is correct semantically (and other SDKs will behave this way, consider SDKs for languages that throw exceptions - there's never a return value to be inspected in that code path).

The correct way forward here is for the S3 service to correct their API modeling. If the delete-marker header is information returned in a 4xx response path, it MUST be included as a member of one of the API's modeled error responses.

RanVaknin commented 3 months ago

Hi @clumio-nick ,

I have raised this as an internal bug report to S3 to fix their model - #P121220645.

Unfortunately this is not actionable by the SDK team, so I'll move it to the cross SDK repo for tracking until we hear back from S3. If you have access to internal support via the developer console, I suggest you reach out as well and add a reference to this ticket ID I provided.

All the best, Ran~

aws / aws-sdk