Unable to use AWS env vars or AWS ECS credentials implicitly when using S3 adapter

[zionsg]

Hi, having an issue where the application is being deployed on AWS ECS Fargate and the access to S3 is defined by the ECS task role, hence no AWS IAM user to issue the usual access key and secret.

In AWS's own Node.js SDK, it is possible to skip the passing in of the access key and secret, and the AWS SDK will automatically read it from a chain of providers, e.g. from env vars or the ECS task role.

However, your library requires the passing in of the AWS credentials at https://github.com/tweedegolf/storage-abstraction/blob/master/src/AdapterAmazonS3.ts

    if (!cfg.accessKeyId || !cfg.secretAccessKey) {
      throw new Error(
        "You must specify a value for both 'applicationKeyId' and  'applicationKey' for storage type 's3'"
      );
    }

Would it be possible to remove this check and let the check be done by the AWS SDK instead?

This flow shows how AWS SDK will automatically search out the credentials from various sources if not provided:

• https://github.com/tweedegolf/storage-abstraction/blob/master/src/AdapterAmazonS3.ts imports S3 from

  • https://github.com/aws/aws-sdk-js/blob/master/clients/s3.js which imports node_loader from

    • https://github.com/aws/aws-sdk-js/blob/master/lib/node_loader.js which defines

      AWS.CredentialProviderChain.defaultProviders = [
        function () { return new AWS.EnvironmentCredentials('AWS'); },
        function () { return new AWS.EnvironmentCredentials('AMAZON'); },
        function () { return new AWS.SsoCredentials(); },
        function () { return new AWS.SharedIniFileCredentials(); },
        function () { return new AWS.ECSCredentials(); },
        function () { return new AWS.ProcessCredentials(); },
        function () { return new AWS.TokenFileWebIdentityCredentials(); },
        function () { return new AWS.EC2MetadataCredentials(); }
      ];        

This flow shows how the AWS S3 client makes use of the credential providers

• https://github.com/aws/aws-sdk-js/blob/master/clients/s3.js imports AWS core from
  • https://github.com/aws/aws-sdk-js/blob/master/lib/core.js which imports Service from
    • https://github.com/aws/aws-sdk-js/blob/master/lib/service.js which creates AWS.Config in initialize()
      • https://github.com/aws/aws-sdk-js/blob/master/lib/config.js which uses credential provider chain from
        • https://github.com/aws/aws-sdk-js/blob/master/lib/credentials/credential_provider_chain.js which uses AWS.CredentialProviderChain.defaultProviders as defined above in https://github.com/aws/aws-sdk-js/blob/master/lib/node_loader.js

[abudaan]

In version 1.4.5 a new key skipCheck has been added to the configuration object. This allows you to let the AWS SDK do the check:

const config = {
  type: StorageType.S3,
  skipCheck: true,
};
const s = new Storage(config);