AWS SDK Requests being recorded on every request

[elopez89]

Description

Pollyjs doesn't seem to replay requests made with the node aws-sdk whenever I run my jest test, the aws request entry inside the recording still seems to be modified. While reviewing the diff between requests the following aws request headers seem to always get updated x-amz-date, date and x-amzn-requestid.

We use AWS Secrets Manager to get API keys, database passwords, and so on. I am sure that it is the aws request because the requests were saved and then replayed successfully when I hardcoded the retrieved secrets and bypassed the request to aws.

Shareable Source

JEST Test

const {
  someAPIRequest,
} = require('../some-api-request');

describe('SomeAPITest', () => {
  it('someEndPoint', async () => {
    const res = await someAPIRequest('someId');

    expect(res).toHaveProperty(['someProperty']);
  });
});

someAPIRequest

async function someAPIRequest(someId){
 // uses aws sdk to retrieve secrets
  const secret = await getSecrets('/some/secret');

  headers.authenticationToken = secret.auth_token;

  return fetch(`some-endpoint`, {
    headers,
  })
    .then(result => result.json())
    .then(result => {
     // perform some business logic on result
     return result;
    })
    .catch(err => {
      console.log(err);
    });
}

module.exports = someAPIRequest;

Jest test setup file

This will run prior to every test.

const setupPolly = require('../lib/setup-polly');

const context = setupPolly();

beforeEach(() => {
  context.polly.server.any().on('beforePersist', (req, recording) => {
        //scrub some headers from requests prior to saving the recordings
    });
  });
});

Error Message & Stack Trace

Whenever I run pollyjs with the option 'recordIfMissing' set to false I see the error 'node-http-adapter' as follows:

=[Polly] [adapter:node-http] Recording for the following request is not found and `recordIfMissing` is `false`.

That makes sense because the file is being changed every time so Pollyjs tries to look for it but it doesn't fit the one that was saved previously.

Config

const path = require('path');
const NodeHttpAdapter = require('@pollyjs/adapter-node-http');
const { Polly } = require('@pollyjs/core');
const FSPersister = require('@pollyjs/persister-fs');
const { MODES } = require('@pollyjs/utils');
const { setupPolly } = require('setup-polly-jest');

Polly.register(NodeHttpAdapter);
Polly.register(FSPersister);

module.exports = () =>
  setupPolly({
    mode: MODES.REPLAY,
    recordIfMissing: process.env.POLLY_RECORD || false,
    adapters: ['node-http'],
    persister: 'fs',
    persisterOptions: {
      disableSortingHarEntries: true,
      fs: {
        recordingsDir: path.resolve(__dirname, '../__recordings__'),
      },
    },
    matchRequestsBy: {
      method: true,
      headers: true,
      body: true,
      order: true,
      url: {
        protocol: true,
        username: false,
        password: false,
        hostname: true,
        port: false,
        pathname: true,
        query: true,
        hash: true,
      },
    },
  });

Dependencies

{   

    "dependencies": {
        "aws-sdk": "2.612.0",
        "node-fetch": "2.6.0"
    },
    "devDependencies": {
        "@pollyjs/adapter-node-http": "5.0.0",
        "@pollyjs/core": "5.0.0",
        "@pollyjs/persister-fs": "5.0.0",
        "@pollyjs/utils": "5.0.0",
        "jest": "24.8.0",
        "setup-polly-jest": "0.9.1"
    }
}

Environment

Node.js v10.15.0
darwin 19.6.0
npm 6.4.1

[offirgolan]

While reviewing the diff between requests the following aws request headers seem to always get updated x-amz-date, date and x-amzn-requestid

Try adding the following to your polly config:

matchRequestsBy: {
   headers: { exclude: ['x-amz-date', 'date', 'x-amzn-requestid', 'authenticationToken'] }
}

[elopez89]

@offirgolan This worked! But unsure why? I was expecting those headers to be removed from the .har file but they are still there. What does this option do? There isn't much information in the documentation other than:

Type: Boolean | Function | Object Default: true

The request headers.

[offirgolan]

The matchRequestsBy config determines how Polly generates the request's unique id so that it can be matched when replayed. The above config tells Polly to ignore those headers when generating the GUID.