lib-storage + S3Client: credentials are not refreshed

artisanbhu commented 1 week ago

Describe the feature

I have some upload code (TS + Electron JS ), where large uploads take a long time to finish - more than one hour. I am using AWS STS assume role to generate AWS session credentials. I can't increase the session expiration hours more than 1 hour because of role chaining issue. The AWS credentials I received from my backend expire after one hour, so I need to refresh them for the large file upload, but I can't figure out how. I tried using the function signature for the configuration field of credentials of the S3Client, but it's not called again after the original credentials expire.

The issue was raised here but it was closed by the author because he switched to use different library. We heavily use aws sdk, we can't afford to switch to another library just for this feature.

Use Case

I use macOS Network Link Conditioner to slow down a large upload, and then I wait more than one hour. After one hour the chunked upload starts failing, but I don't see a new call to credentials: async (): Promise.

import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
import type { AwsCredentialIdentity } from "@aws-sdk/types/dist-types/identity/AwsCredentialIdentity";

// ...

const selectedFile: File = //...
const bucket: string = //...
const region: string = // ...
const key: string = // ...
const multipartUpload = new Upload({
  client: new S3Client({
    maxAttempts: 10,
    region,
    credentials: async (): Promise<AwsCredentialIdentity> => {
      console.log("REFRESHING CREDENTIALS!");
      const session = await getNewSessionFromBackend(); 
      const { accessKeyId, secretAccessKey, sessionToken } = session.uploadCredentials;
      return {
        accessKeyId,
        secretAccessKey,
        sessionToken,
      };
    },
  }),
  params: { Bucket: bucket, Key: key, Body: selectedFile },
});

// Irrelevant, but for completeness:
multipartUpload.on("httpUploadProgress", handleUploadProgress);
multipartUpload.done().then(handleUploadResponse).catch(handleUploadError);

Proposed Solution

As discussed here we should have a way to refresh the s3 client after session expires in 1 hour but large file multipart upload should continue without any error.

Other Information

No response

Acknowledgements

[ ] I may be able to implement this feature request
[ ] This feature might incur a breaking change

SDK version used

"@aws-sdk/client-s3": "^3.540.0", "@aws-sdk/lib-storage": "^3.572.0",

Environment details (OS name and version, etc.)

Electron JS App

RanVaknin commented 1 week ago

Hi @artisanbhu ,

The SDK's default credential chain should automatically refresh credentials whenever they are close to expiration. The cache will be checked every time an API operation is going to be called.

Meaning that every time the SDK will call uploadPart() (lib-storage's upload() calls this method under the hood) the SDK will check the client's credential cache to see if the credentials are expired or about to expire and depending on the method you defined for credential retrieval it will re-attempt to get those credentials.

However, Because your credential object does not have an expiration field on it, the SDK will treat it as static credentials and will not to attempt to refresh it.

This worked for me:

const client = new S3Client({
    region: "us-east-1",
    credentials: async () => {
      console.log("REFRESHING CREDENTIALS!");
      const { accessKeyId, secretAccessKey, sessionToken } = await uploadCredentials();
      return {
        accessKeyId,
        secretAccessKey,
        sessionToken,
        expiration: new Date(Date.now() + 300_000),
      };
    },
})

REFRESHING CREDENTIALS!
Uploaded 5242880 out of 1073741824
REFRESHING CREDENTIALS!
Uploaded 10485760 out of 1073741824
REFRESHING CREDENTIALS!
Uploaded 15728640 out of 1073741824
REFRESHING CREDENTIALS!
Uploaded 20971520 out of 1073741824
REFRESHING CREDENTIALS!
Uploaded 26214400 out of 1073741824
REFRESHING CREDENTIALS!
Uploaded 31457280 out of 1073741824
REFRESHING CREDENTIALS!
...

Thanks, Ran~

artisanbhu commented 1 week ago

Thank you so much for the suggestion. Also, this link was useful.

While applying your suggested solution I found out that credential provider is called again once the client detects that the credentials are within 5 minutes of expiring. If you are setting expiration time less than or equal to 5 minutes the credential provider will be called infinitely.

aws / aws-sdk-js-v3