[QUESTION] - Do anyone have some example of testing with dynamodb and dynogles

[sp90]

Hi everyone,

Im looking for a solution to start testing my backend solution, and i would love to see some examples on how to test my functions that call the database, but instead of calling the real database then do a mock CRUD thing og similar

[ghost]

This is opinionated example using jest.
We're using typescript and node on the project btw...
Notice the style we set up the persistence, it a bit different than you'll find in the examples of dynogels usage. We're using it more in spring jpa style having repository classes separate from the model. It's just a another convenient level of abstraction which is supported by dynogels.
I've exposed here the wider context of usage to give people idea about possible solution for separation of concerns when using dynogels.
Main point for you is that you should completely mock aws-sdk and dynamoDB modules. Jest helps us there by it's automocking feature, i'd love to hear other ideas too.

Abstract class

import * as dynogels from 'dynogels';
import {Model, ModelConfiguration, Scan} from "dynogels";

export abstract class BaseRepository {

  protected _model: Model;

  constructor() {
  }

  abstract readonly _TABLE: string;

  abstract readonly DYNOGELS_CONFIG: ModelConfiguration;

  protected dynogelsDefine(): void {
    this._model = dynogels.define(this.constructor.name, this.DYNOGELS_CONFIG);
  }

  public query(): Scan {
    return this._model.scan();
  }

  get model(): Model {
    return this._model;
  }

  get table(): string {
    return this._TABLE
  }
}

Concrete implementation

import {Person} from "entities";
import {BaseRepository} from "./base.repository";
import * as dynogels from "dynogels";
import * as Joi from 'joi';

export class PersonRepository extends BaseRepository {

  constructor() {
    super();
    this.dynogelsDefine();
  }

  readonly _TABLE: string = 'matches';

  readonly DYNOGELS_CONFIG: dynogels.ModelConfiguration = {
    hashKey: 'id',
    tableName: this.table,
    schema: {
      id: Joi.string(),
      firstName: Joi.number(),
      lastName: Joi.string(),
    }
  };

  getById(id): Promise<Person> {
    // We need a feature in dynogels Models to return promisses from all Model methods
    return new Promise((resolve, reject) => {
      this._model.get(id, {ConsistentRead: true}, (err, result) => {
        if (err) {
          reject(err);
        } else {
          // We need a feature in dynogels to avoid this
          let response = Object.is(null, result) ? null : (result as any).attrs;
          resolve(response);
        }
      });
    });
  }

  getAll(): Promise<Array<Promise>> {
    // We need a feature in dynogels Models to return promisses from all Model methods
    return new Promise((resolve, reject) => {
      this._model
        .scan()
        .exec((err, result) => {
          if (err) {
            reject(err);
          } else {
            // We need a feature in dynogels to avoid this
            let response = (result) ? (result as any).Items.map(result => result.attrs) : [];
            resolve(response);
          }
        })
    });
  }
}

Mocks

module.exports = {
  'aws-sdk': jest.genMockFromModule('aws-sdk') as any,
  DynamoDB : jest.genMockFromModule('aws-sdk/clients/dynamodb')
};

Example unit test for one such repository

describe('Test PersonRepository', () => {

  test('Constructor returns an instance of PersonRepository', () => {
    expect(repository).toBeInstanceOf(PersonRepository);
  });

  test('Dynogels model is defined on instance of PersonRepository', () => {
    expect(repository.model).toBeDefined();
  });

});

[sp90]

@maljukan thanks, great example, im going to try https://www.npmjs.com/package/aws-sdk-mock to mock responses and then test it that way

[ghost]

@sp90 Great, please share your findings. My approach is kind a different. I'm thinking about manually mocking dynogels calls to aws-sdk but i'm not certain at the moment if that's the right way to go.

[sp90]

@maljukan i will share, and i would love to see other approaches out there?

[cdhowie]

We do service-level testing with dynalite. The test harness deletes the test database on each run and then launches dynalite, creates all of the tables, launches the REST API server, and then begins the service tests. This is nice for making sure that the whole system works in the presence of a functioning database backend; e.g. after a create REST operation there is a get-list and get-one test to make sure the REST server returns the object that was just created.

[cdhowie]

Since this is kind of an open-ended discussion, I'm going to close the issue since a few ideas have already been given. Don't let this stifle any discussion; I just don't want this issue showing up as open forever.

[sp90]

@cdhowie AMAZING with dynalite, thats exactly what i was looking for 👍

[wuichen]

@cdhowie @sp90 do you guys have any example of how to set dynogels and dynalite up fir testing?

[cdhowie]

@wuichen This is close to what we do for dynalite:

function createDynalite() {
    return new Promise((resolve, reject) => {
        let db = dynalite({
            path: './dev-db',
            createTableMs: 0
        });

        db.listen(4567)
        .on('listening', () => resolve(db))
        .on('error', reject);
    });
}

createDynalite()
.then(db => { dynogels.dynamoDriver(db); })
.then(/* call dynogels.createTables(), start your application */);

If you do dynogels.createTables({ $dynogels: { pollingInterval: 10 } }) then the operation will return much quicker. I would not suggest using this argument against the real DynamoDB, as it will result in a storm of HTTPS calls. When configured with createTableMs: 0, Dynalite creates tables much faster than DynamoDB and so it will make your test startup time much faster if you tell dynalite that it can query for table creation completion at a much faster rate.