does this work with parallel tests?

[sibelius]

I'd like to have a different database for each of my test

this setup make this possible?

[vladholubiev]

@sibelius would you like a different version of mongodb for each test or just a fresh empty db? The latter is possible and makes sense

[sibelius]

An empty dB for each test

[vladholubiev]

@sibelius if you're ok with empty db for each test file - it's already like that now. If empty db for every test - then you should add const col = db.getCollection('collection_name'); col.deleteMany(); inside beforeEach() hook in test file

[sibelius]

we are using a different approach like in this medium post https://itnext.io/parallel-testing-a-graphql-server-with-jest-44e206f3e7d2

we don't have a global teardown, and we create a MongodbMemoryServer per each node environment

does that makes sense? or we should just have one MongodbMemoryServer?

cc @nodkz

[sibelius]

another problem is looks like mongoose is leaking

https://github.com/facebook/jest/issues/3602#issuecomment-361477979

[nodkz]

@sibelius MongodbMemoryServer designed to run in parallel. But it cost you for every instance about 10mb of RAM and several seconds for spin up a binary file.

In my cases I'm using one MMS per test-file. It helps me isolate parallel execution by providing separate DB for every test-file.

But inside one test file for me is faster to cleanup collections, than spin up new MMS.

[sibelius]

we figure it out the memory leak, it was on graphql-redis-subscriptions

so our tests are working fine on parallel

this setup does not work well with parallel to make work on parallel we create a custom jest environment

/* eslint-disable */
const MongodbMemoryServer = require('mongodb-memory-server');
const NodeEnvironment = require('jest-environment-node');

class MongoDbEnvironment extends NodeEnvironment {
  constructor(config) {
    // console.error('\n# MongoDB Environment Constructor #\n');
    super(config);
    this.mongod = new MongodbMemoryServer.default({
      instance: {
        // settings here
        // dbName is null, so it's random
        // dbName: MONGO_DB_NAME,
      },
      binary: {
        version: '4.0.0',
      },
      // debug: true,
      autoStart: false,
    });
  }

  async setup() {
    await super.setup();
    // console.error('\n# MongoDB Environment Setup #\n');
    await this.mongod.start();
    this.global.__MONGO_URI__ = await this.mongod.getConnectionString();
    this.global.__MONGO_DB_NAME__ = await this.mongod.getDbName();
  }

  async teardown() {
    await super.teardown();
    // console.error('\n# MongoDB Environment Teardown #\n');
    await this.mongod.stop();
    this.mongod = null;
    this.global = {};
  }

  runScript(script) {
    return super.runScript(script);
  }
}

module.exports = MongoDbEnvironment;

tks for the help

[jardakotesovec]

@sibelius @vladgolubev I am still confused (and I read several related issues) why the current setup in this repo is supposed to work fine for parallel testing. setup.js is run once before test files are run - therefore there is just once instance of mongod-memory which is shared by all test files. So how the separation between test files is working? I would expect that with this setup every test file is talking to same database, therefore there is many opportunities for race conditions.

[sibelius]

@jardakotesovec check this https://itnext.io/parallel-testing-a-graphql-server-with-jest-44e206f3e7d2

and we have a running example of this approach here https://github.com/entria/graphql-dataloader-boilerplate

we have one database per test, so we can run them in parallel

[jardakotesovec]

@sibelius Thanks. That makes sense - if the mongod is created in env file that I understand that you will get independent db per database.

My confusion was really from this comment which sounds that with the setup which is in this repo you can do parallel testing and each file has independent database.. So I assume thats not the case and example in this repo is sharing one database across all test file, which is usually not desirable I would think.

[Yankovsky]

@sibelius @jardakotesovec @vladgolubev Isn't it better to run one mongo server and create a database for each test? This way tests will be isolated and there will be no need to restart mongo server each time.

[Yankovsky]

We can use uuid or normalized path to test file as db name to ensure its uniqueness

[sibelius]

check our setup here https://github.com/entria/entria-fullstack/blob/master/packages/server/test/environment/mongodb.js

[Yankovsky]

@sibelius Yeah, I got your idea. You start new MongodbMemoryServer for each test. I thought that we could start only one MongodbMemoryServer and create different db for each test. But I've just checked MongodbMemoryServer source and it seems like for one MongodbMemoryServer there could be only one db https://github.com/nodkz/mongodb-memory-server/blob/master/src/MongoMemoryServer.ts#L68

[sibelius]

open an issue there

[Yankovsky]

@sibelius I'll try your approach first. Do you have a lot of tests? Does creating new MongodbMemoryServer works fast enough? Thanks for help!

[sibelius]

5k ~ 8minutes

[nodkz]

@sibelius @Yankovsky it's by design: Class MongodbMemoryServer creates only one DB. But you may create many MongodbMemoryServer instances. 😂

[nodkz]

Read the docs https://github.com/nodkz/mongodb-memory-server#several-mongoose-connections-simultaneously

[Yankovsky]

@nodkz if I understand correctly, we need to create new MongodbMemoryServer for each test for proper isolation. In @sibelius's solution new MongodbMemoryServer is created for each test file but not for each test.

[sibelius]

we clean database before each test inside the same test file

[nodkz]

@Yankovsky you MAY create MongodbMemoryServer for every test. But it will be performance degradation. Better to create one MongodbMemoryServer per test file.

Anyway if you have complex solutions, you may create in one test file several MongodbMemoryServer.

I agree with @sibelius solution. He chooses the right strategy.

[danny-does-stuff]

Leaving this here because it would have saved me hours if it was on this thread

The docs imply that using instance: {} in the config file will use a new database name for every test file, however I was not seeing this behavior when using process.env.MONGO_URL.

I switched from

// OLD CODE, USES SAME DB FOR EVERY TEST

beforeAll(async () => {
    await mongoose.connect(process.env.MONGO_URL, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useCreateIndex: true,
    })
})

to

// NEW CODE, ACTUALLY USES THE UNIQUE DB NAME FOR EACH TEST FILE
/**
 * Gets the mongo url by combining variables in `process.env` and `global`. Uses the mongo server host
 * and port from process.env.MONGO_URL, and the db name from `global.__MONGO_DB_NAME_`.
 */
function getMongoUrl() {
    // Replace the db name to use a unique db name for each test
    return (
        process.env.MONGO_URL.split('/')
            .slice(0, -1)
            .join('/') + `/${global.__MONGO_DB_NAME__}`
    )
}

beforeAll(async () => {
    await mongoose.connect(getMongoUrl(), {
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useCreateIndex: true,
    })
})

It seems like using instance: {} should set the unique db name in process.env.MONGO_URL automatically, but perhaps there's a reason it does not do that.

inspired by comment by @sbland here