aws / aws-sdk

Landing page for the AWS SDKs on GitHub
https://aws.amazon.com/tools/
Other
68 stars 12 forks source link

Usage of Tier in `S3.ClientRestoreObject` #713

Closed wunderbarb closed 3 months ago

wunderbarb commented 3 months ago

Describe the issue

In the structure s3.RestoreObjectInput.RequestRestore, there are two times the types.Tier. The first one within RestoreRequest.GlacierJobParameters and the one as RestoreRequest.Tier.
1- What is the difference? What would happen if they differ? 2- GlacierJobParameteters is not valid when using OutputLocation. Thus, I guess that we should use the second one to define the retrieval speed. Unfortunately when using something like

        Bucket: aws.String(b),
        Key:    aws.String(p),
        RestoreRequest: &types.RestoreRequest{
                Tier: types.TierExpedited,

        },
    }
    if oo.restoreLocation != "" {
        br, kr := parse1(oo.restoreLocation)
        roIN.RestoreRequest.OutputLocation = &types.OutputLocation{
            S3: &types.S3Location{
                BucketName: br,
                Prefix:     kr,
            },
        }
    } else {
        roIN.RestoreRequest.Days = aws.Int32(days)
    }
``` creates an XML misformation whereas ```
roIN := &awsS3.RestoreObjectInput{
        Bucket: aws.String(b),
        Key:    aws.String(p),
        RestoreRequest: &types.RestoreRequest{
            GlacierJobParameters: &types.GlacierJobParameters{
                Tier: types.TierExpedited,
            },
        },
    }
    if oo.restoreLocation != "" {
        br, kr := parse1(oo.restoreLocation)
        roIN.RestoreRequest.OutputLocation = &types.OutputLocation{
            S3: &types.S3Location{
                BucketName: br,
                Prefix:     kr,
            },
        }
    } else {
        roIN.RestoreRequest.Days = aws.Int32(days)
    }``` works.  
    THus, how can I define the "speed": bulk, standard or expedited when using `OutputLocation`?

### Links

https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/s3@v1.53.0/types#RestoreRequest

### AWS Go SDK V2 Module Versions Used
github.com/aws/aws-sdk-go-v2 v1.26.0
github.com/aws/aws-sdk-go-v2/config v1.26.6
github.com/aws/aws-sdk-go-v2/credentials v1.16.16
github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue v1.12.15
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.13
github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs v1.31.0
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.26.9
github.com/aws/aws-sdk-go-v2/service/eventbridge v1.28.0
github.com/aws/aws-sdk-go-v2/service/lambda v1.49.7
github.com/aws/aws-sdk-go-v2/service/s3 v1.53.0
github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.26.2
github.com/aws/aws-sdk-go-v2/service/sesv2 v1.24.6
github.com/aws/aws-sdk-go-v2/service/sns v1.26.7
github.com/aws/aws-sdk-go-v2/service/sqs v1.31.3
github.com/aws/aws-sdk-go-v2/service/ssm v1.49.1
github.com/aws/aws-sdk-go-v2/service/textract v1.28.6
RanVaknin commented 3 months ago

Hi @wunderbarb ,

Thanks for reaching out. The restore operation is notorious for lacking documentation. I personally have seen many other customers running into the same issue and I reached out to S3 internally trying to get them to fix their docs but to date this has not happened.

related https://github.com/aws/aws-sdk/issues/505

From my correspondence with the S3 team:

For the two types of tier parameter, the GlacierJobParameters tier is used for normal restore operations, while the Tier on the root request is only used for select restore operations (which is currently broken for newly created objects).

Which leads me to believe that OutputLocation should only be used for select restore and not glacier restore (normal restore)

I have reached out to the S3 team again for clarification on your behalf.

Thanks, Ran~

wunderbarb commented 3 months ago

Thanks.
Your answer seems to indicate that OutputLocation cannot be used in "normal" operations. I was expecting to use it to restore directly in a given location with additional custom contextual User Metadata. Thus, it seems that I have to design a different solution.
Let's wait for the S3 team's confirmation.

RanVaknin commented 3 months ago

Since this is not a Go SDK problem, Im transferring this to the cross SDK repo for tracking. Internal ticket with S3 - P124047747

RanVaknin commented 3 months ago

Hi @wunderbarb

I heard back from the S3 service team. The following inputs are only for glacier select jobs:

Thanks, Ran~

github-actions[bot] commented 3 months ago

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.