Help: How to use AWS-SDK-MOCK with S3 upload?

[wontwon]

Hi all, after reviewing the docs, I'm still having a hard time understanding exactly how to mock out S3 functionality. In this case, I want to mock uploading a file and then writing to dynamoDB.

Any help would be greatly appreciated here.

[finferflu]

I'm not an expert, but I guess providing some examples of what you've tried should help others to assist you.

[hoegertn]

Hi, to mock S3.upload I use the following code. Maybe this helps.

    AWS.mock('S3', 'upload', (params, callback) => {
      params.should.be.an.Object();
      params.should.have.property('Bucket', 'TestBucket');
      params.should.have.property('Key');
      params.should.have.property('Body', 'TestBuffer');

      callback(null, {
        ETag: 'SomeETag"',
        Location: 'PublicWebsiteLink',
        Key: 'RandomKey',
        Bucket: 'TestBucket'
      });
    });

[sfradkin]

I'm trying to mock an S3 upload but the upload is being performed by the multer-s3 library. In my case, mocking the upload as in the above comment results in an error: upload.on is not a function. The multer-s3 library is trying to call the 'on' function and the 'send' function on the return value from 'upload'. Not quite sure where to go next to get this to work.

[nkov]

@sfradkin The way we solved this was by stubbing multer directly here, which was a pain because it exports a default function. So we created a local module to wrap multer, and then stubbed that with sinon:

modules/multer.js

const multer = require('multer')
module.exports = { multer }

service.js

const MulterWrapper = require('./modules/multer.js')
const upload = MulterWrapper.multer({....})

service.postFile = [
   upload.single('file'),
   (req, res, next) => { ... }
]

test.js

const MulterWrapper = require('../modules/multer')
sinon.stub(MulterWrapper, 'multer').returns(multer({ storage: multer.memoryStorage() }))

The reason to mock it as memoryStorage is so that you can still access the file in the request object.

[jhalborg]

I'm having a hard time getting this to work this async/await in Typescript.

I want to test the following method:

export async function uploadImage(
  base64image: string
): Promise<{ original: string; }> {
  const base64Image = Buffer.from(base64image.replace(/^data:image\/\w+;base64,/, ''), 'base64');

  // Create S3 service for upload
  const service = new aws.S3({
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    region: process.env.AWS_REGION,
    apiVersion: '2006-03-1', // Is the latest version according to docs https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html
  });

  const id = uuid.v4();

  const sharedImageParams = {
    Bucket: process.env.AWS_IMAGE_BUCKET_NAME,
    ACL: 'public-read',
    ContentEncoding: 'base64',
    ContentType: `image/${type}`,
  };

  const imageParams: aws.S3.PutObjectRequest = {
    Key: `${id}:${ImageResizeOptions.variantNames.original}.${type}`,,
    Body: base64Image,
    ...sharedImageParams,
  };

  const uploadOriginal = await service.upload(imageParams).promise();

  return {
    original: uploadOriginal.Location,
  };

Trying to test it with:

import AWS_MOCK from 'aws-sdk-mock';
import * as aws from 'aws-sdk';

dotenv.config();

jest.mock('aws-sdk');

describe('utility/imageUploadHelper', () => {
  beforeAll(() => {
    process.env.AWS_ACCESS_KEY_ID = 'keyID';
    process.env.AWS_SECRET_ACCESS_KEY = 'key';
    process.env.AWS_REGION = 'region';
    process.env.AWS_IMAGE_BUCKET_NAME = 'Bucket';
  });

  afterAll(() => {
    AWS_MOCK.restore('S3');
    dotenv.config();
  });

  it('constructs an AWS sdk instance for upload', async () => {
    const fakeUpload = jest.fn(async () => {
      return new Promise<aws.S3.ManagedUpload.SendData>(resolve => {
        resolve({
          ETag: 'SomeETag',
          Location: 'PublicWebsiteLink',
          Key: 'RandomKey',
          Bucket: 'TestBucket',
        });
      });
    });

    AWS_MOCK.mock('S3', 'upload', (params, callback) => {
      callback(null, fakeUpload(params));
    });

    const res = await resizeAndUploadImage(base64image);
    expect(fakeUpload).toHaveBeenCalledTimes(1);
  });
});

But I keep getting TypeError: Cannot read property 'promise' of undefined on the service.upload call. The weird thing is, when debugging the test, I can see that service is correctly a mockConstructor and upload is a mock function, and the imageParams look as expected.

Maybe the promise() method is not implemented in this library?

I also tried the following, but the mock function fakeUpload is never called no matter what I do

it('constructs an AWS sdk instance for upload', async () => {
    const fakeUpload = jest.fn(() => {

      const asd: aws.S3.ManagedUpload = {
        promise: jest.fn(() => {
          console.log(TAG, `promise\n`);
          const result: aws.S3.ManagedUpload.SendData = {
            Bucket: 'Bucket',
            ETag: 'ETag',
            Key: 'Key',
            Location: 'Location',
          };
          return result;
        }),
        abort: jest.fn(() => {
          console.log(TAG, `abort\n`);
        }),
        send: jest.fn(() => {
          console.log(TAG, `send\n`);
        }),
        on: jest.fn(() => {
          console.log(TAG, `on\n`);
        }),
      };
      return asd;
    });

    AWS_MOCK.mock('S3', 'upload', fakeUpload);

    const res = await resizeAndUploadImage(base64image);
    expect(fakeUpload).toHaveBeenCalledTimes(1);
  });

[lcardito]

Came across this one, no typescript for me however when mocking S3 we need to make sure we consume the fileStream sent to the lib, otherwise busboy won't complete the upload.

So what I did is expose the S3 object in a module (not eve sure if needed..)

module.exports = new aws.S3({ credentials: null, credentialProvider: awsCredProviderChain });

Then stub it with Sinon:

    sinon
        .stub(s3Provider, 'upload')
        .callsFake((...opts) => {
            let fileStream = opts[0]['Body'];
            return {
                on: (type, callback) => {
                    callback({total: 416});
                },
                send: callback => {
                    fileStream.resume();
                    callback(null, {
                        'Location': 'mock-location',
                        'ETag': 'mock-etag',
                        'VersionId': 'mock-version-id'
                    });
                }
            };
        });

[ToddRuan0]

For me, this work

const AWS = require('aws-sdk');

    let stream;
    sinon
      .stub(AWS.S3.prototype, 'upload')
      .callsFake((...params) => {
        stream = params[0].Body;
        return {
          send(callback) {
            callback(null, { body: { status: 'ok' } });
          },
        };
      });