MIGRATION ISSUE: Cannot Upgrade S3.ManagedUpload() to Upload()

rgare-travistrue commented 1 month ago

Pre-Migration Checklist

[X] I've read the Migration Guide.
[X] I've reviewed the upgrading notes and major version differences mentioned in UPGRADING.md.
[X] I've checked AWS Forums and StackOverflow for similar migration issues.

Which JavaScript Runtime is this issue in?

Browser

AWS Lambda Usage

[X] Yes, my application is running on AWS Lambda.
[ ] No, my application is not running on AWS Lambda.

Describe the Migration Issue

I'm running into an issue with updating Cognito authorization, so that our React app can upload objects directly to S3. I attempted to translate the call to AWS.CognitoIdentityCredentials() to a set of v3 calls.

Code Comparison

Here's the working v2 snippet:

export async function authorize () {
  const session = await Auth.currentSession()
  const token = session.getIdToken().getJwtToken()

  AWS.config.credentials = new AWS.CognitoIdentityCredentials(
    {
      IdentityPoolId: process.env.IDENTITY_POOL_ID,
      RoleArn: process.env.USER_UPLOAD_ROLE_ARN,
      Logins: {
        [process.env.USER_POOL_COGNITO_ID]: token
      }
    },
    {
      region: 'us-east-1'
    }
  )
}

const uploadFile = (file, id, country) => {
  await authorize()

  const { region } = apiHelper.getCountry(country)

  const guid = cuid()
  file.guid = guid

  const upload = new S3.ManagedUpload({
    params: {
      Bucket: `${apiHelper.getBucketName(country)}`,
      Key: `${id}/${guid}/${file.name}`,
      Body: file
    },
    service: new S3({ region })
  })

  await upload
    .on('httpUploadProgress', (progress) => {})
    .promise()
}

export default { uploadFile }

Here's the working v3 snippet:

const identityClient = new CognitoIdentityClient(CLIENT_CONFIG)
const stsClient = new STSClient(CLIENT_CONFIG)

async function buildS3Client (country) {
  const session = await Auth.currentSession()
  const token = session.getIdToken().getJwtToken()
  const { region } = apiHelper.getCountry(country)

  const { IdentityId } = await identityClient.send(new GetIdCommand({
    IdentityPoolId: process.env.IDENTITY_POOL_ID,
    Logins: {
      [process.env.USER_POOL_COGNITO_ID]: token
    }
  }))

  const openIdTokenRes = await identityClient.send(new GetOpenIdTokenCommand({
    IdentityId,
    Logins: {
      [process.env.USER_POOL_COGNITO_ID]: token
    }
  }))

  const openIdToken = openIdTokenRes.Token

  const { Credentials } = await stsClient.send(
    new AssumeRoleWithWebIdentityCommand({
      RoleArn: process.env.USER_UPLOAD_ROLE_ARN,
      RoleSessionName: 'app1', // TODO: add better session control before PR
      WebIdentityToken: openIdToken
    })
  )

  console.log('Credentials:', Credentials)

  return new S3Client({
    region,
    credentials: Credentials
  })
}

const authorize = () => {
  /* TODO: fix this */
}

const upload = (file, id, country) => {
  file.guid = crypto.randomUUID()

  const instance = new Upload({
    client: await buildS3Client(country),
    params: {
      Bucket: apiHelper.getBucketName(country),
      Key: `${id}/${file.guid}/${file.name}`,
      Body: file
    }
  })

  await instance
    .on('httpUploadProgress', progress => {})
    .done()
}

Observed Differences/Errors

I tried replacing AWS.CognitoIdentityCredentials() with fromCognitoIdentityPool(), but got a 403 Access Denied error from S3's API when I used it because there's no way to provide a RoleArn like we can do with the old AWS.CognitoIdentityCredentials().

I old AWS.CognitoIdentityCredentials() appears to be a wrapper function around multiple Cognito calls, depending on the parameters that are provided. I ended up calling the v3-equivalent calls that make up AWS.CognitoIdentityCredentials() to replicate the RoleArn using the STS Client's AssumeRoleWithWebIdentityCommand(), but now I get this error:

Error: Resolved credential object is not valid
    at SignatureV4S3Express.validateResolvedCredentials (SignatureV4.js:180:1)
    at SignatureV4S3Express.signRequest (SignatureV4.js:105:1)

Additional Context

Here's the shape of the Credentials object that I get back from my call to AssumeRoleWithWebIdentityCommand():

{
    "AccessKeyId": "********************",
    "SecretAccessKey": "****************************************",
    "SessionToken": "***.***.***",
    "Expiration": "2000-01-01T00:00:00.000Z"
}

I looked at the source code for the error, and it's because all of the property names in Credentials are in PascalCase and not camelCase. The S3Client constructor is expecting credentials with camelCase property names. Rebuilding a new object with the expected-named property names seemed to have solved my problem:

return new S3Client({
  region,
  credentials: {
    accessKeyId: Credentials.AccessKeyId,
    secretAccessKey: Credentials.SecretAccessKey,
    sessionToken: Credentials.SessionToken,
    expiration: Credentials.Expiration
  }
})

aBurmeseDev commented 1 month ago

Hi @rgare-travistrue - thanks for reaching out and sorry for the inconvenience.

I tried replacing AWS.CognitoIdentityCredentials() with fromCognitoIdentityPool(), but got a 403 Access Denied error from S3's API when I used it because there's no way to provide a RoleArn like we can do with the old AWS.CognitoIdentityCredentials().

Regarding the issue with the credential provider: You mentioned that you tried replacing AWS.CognitoIdentityCredentials() with fromCognitoIdentityPool(), but encountered a 403 Access Denied error from the S3 API when using it. The reason for this error is that there is no way to provide a RoleArn with fromCognitoIdentityPool(), which was possible with the old AWS.CognitoIdentityCredentials().

To resolve this, you should be able to use the customRoleArn option when calling fromCognitoIdentityPool() to provide the required RoleArn.

Concerning the second issue: You seem to be directly passing STS credentials to the SDK, which returns credentials in camel case format. However, the SDK expects the credentials to be in lower case format. This mismatch in casing is the expected behavior when passing STS credentials directly to the SDK.

return new S3Client({
  region,
  credentials: {
    accessKeyId: Credentials.AccessKeyId,
    secretAccessKey: Credentials.SecretAccessKey,
    sessionToken: Credentials.SessionToken,
    expiration: Credentials.Expiration
  }
})

Best, John

rgare-travistrue commented 1 month ago

Hey John, thanks for the reply. I also tried passing the ARN as the customRoleArn property in fromCognitoIdentityPool(), but then I got an error stating something about Cognito missing ARN pairs or something similar to that. I'm not too familiar with Cognito, but it seems like upgrading to v3 requires us to do some additional configuration with our Cognito user pool.

Anyway, this workaround seemed to do the trick for now. Our team was going to replace the AWS SDK with Amplify anyway, so the overall goal was to get a quick fix in for now since we're going to swap out the AWS SDK in the next few months anyway.

github-actions[bot] commented 1 month ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

aws / aws-sdk-js-v3