@aws-sdk/credential-provider-ini - Enable uncached request for credentials

[mdpratt]

Is your feature request related to a problem? Please describe.

As a user, I want want to automatically refresh credentials in my service if it detects a change in credentials. Invoking fromIni() will always return the same results, even if the ~/.aws/credentials file has changed between invocations.

Describe the solution you'd like

I want to be able to retrieve current credentials, regardless of how many times I've invoked fromIni(). This could be an extra { ignoreCache: true } property that I can provide.

Describe alternatives you've considered

I've tried using @aws-sdk/shared-ini-file-loader directly, and deleting the require / import but that apparently won't clear the const fileStatusHash: { [key: string]: FileStatus } = {}; object.

// Example, assume wrapped in a function
delete require.cache[require.resolve('@aws-sdk/shared-ini-file-loader')]
let loader = require('@aws-sdk/shared-ini-file-loader');
loader.loadSharedConfigFiles().then((p: any) => console.log(p.credentialsFile['default']))

[mdpratt]

After thinking some more, since the fileStatusHash just uses the filename as the key, I can get around this problem by creating and then deleting temporary symlinks. Full example below, when run, will watch the ~/.aws/credentials file and then console.log the default profile. This does run the risk of an eventual memory leak, but it's okay for my shorter lived process.

import chokidar from 'chokidar';
import fs from 'fs';
import { homedir } from 'os';
import { resolve } from 'path';
import { nanoid } from 'nanoid';
import { fromIni } from '@aws-sdk/credential-providers';

const { symlink, unlink } = fs.promises;
const AWS_CREDENTIALS = resolve(`${homedir}/.aws/credentials`);

const getCredentials = () => {
  const filepath = `./${nanoid(10)}`
  symlink(AWS_CREDENTIALS, filepath, 'file')
    .then(() => fromIni({ filepath })())
    .then(console.log) // Or do whatever
    .then(() => unlink(filepath));
}

chokidar.watch(AWS_CREDENTIALS).on('change', getCredentials)

[Westermann]

:+1: I also think I need this!

[jaypan13]

I am also using @aws-sdk/shared-ini-file-loader and this kind of property will be definitely helpful in my case when my creds from credentials file are added/modified using another application from backend and my frontend is receiving only cached credentials.

[JoshuaWise]

We are running into this issue here at Postman, forcing us to use the old v2 SDK.

[Aeolun]

@ajredniwja Is there anything that needs to be done so that this PR can be merged?

[bboure]

I am facing the same problem.

If that helps, I found another workaround using v2.

import { SharedIniFileCredentials } from 'aws-sdk';

export const fromIni = (params) => {
  return async () => {
    const credentials = new SharedIniFileCredentials(params);

    return {
      accessKeyId: credentials.accessKeyId,
      secretAccessKey: credentials.secretAccessKey,
      sessionToken: credentials.sessionToken,
    };
  };
};

[stoyan-scava]

Thanks @bboure I can confirm using SharedIniFileCredentials from sdk v2 works!

[vecerek]

Feature parity with v2 would be nice:

Occasionally credentials can expire in the middle of a long-running application. In this case, the SDK will automatically attempt to refresh the credentials from the storage location if the Credentials class implements the {refresh} method.

https://github.com/aws/aws-sdk-js/blob/5c14d6b1447a44454189bacd1ab5351433746a4e/lib/credentials.js#L16-L19

[vecerek]

@bboure I've tried your workaround and after 12 hours, the credentials expired and I'm getting ExpiredToken: The provided token has expired. errors. Is this expected, or am I supposed to implement something extra to ensure that the tokens get refreshed? I have a process set up that makes sure the credentials file is updated just before the token expires.

[MartinDevillers]

I am running into the same issue on my local development machine where I use oktacli to automatically refresh the credentials file each hour. The current implementation of @aws-sdk/shared-ini-file-loader treats the files as static and there's no way to clear the SDK's internal cache without a full NodeJS restart. It'd be great if there's a way to prevent the file-loader from caching the files, or even better, if it would watch the file and refresh automagically when the contents change.

[bboure]

@vecerek I added this in the return clause to solve the same problem:

expiration:
        credentials.expireTime ||
        DateTime.now().plus({ minutes: 5, seconds: 30 }).toJSDate(),

explanation: If expiration is not present, clients assume the tokens are always valid and they won't call fromIni again. When expiration is present, fromIni is called again 5 minutes before expiry. In my case, I wanted to force a refresh every 30 seconds, so I set a fake expiry date of 5 min and 30 seconds.

[edgmark]

Hi, has any work been done to handle this scenario officially, or are the workarounds still required?

Edit: I see in the linked MR that there is now an ignoreCache property, so I'm going to try that. Assuming that works, it would be nice if that were added to the docs: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-credential-providers/#fromini

[edgmark]

I tried ignoreCache, and I'm still getting errors about expired tokens after 1 hour. This is abbreviated, but hopefully gets the point across. I've confirmed that the underlying ini file does have new credentials, which are refreshed via an external process.

// Set globally:
const credentials = fromIni({
    ignoreCache: true,
    profile: "app"
})
export const client = S3Client({
    credentials: awsKeysRefresher.credentials,
    region: e2eAwsConfig.s3Region
})

// And on a 45 minute interval:
const refreshedCreds = fromIni({
    ignoreCache: true,
    profile: "app"
})
client.config.credentials = refreshedCreds