shelfio / jest-mongodb

Jest preset for MongoDB in-memory server
MIT License
591 stars 83 forks source link

Exception when using change streams #260

Open jmbldwn opened 3 years ago

jmbldwn commented 3 years ago

If I subscribe to a change stream, using mongoose's watch method, I encounter the following exception.

Is there a way to configure the mongo memory server to support this?

Exception has occurred: Error [ERR_UNHANDLED_ERROR]: Unhandled error. (MongoError: Storage engine does not support read concern: { readConcern: { level: "majority" } }
  at MessageStream.messageHandler (/Users/jim/development/.../node_modules/mongodb/lib/cmap/connection.js:263:20)
    at MessageStream.emit (events.js:315:20)
    at processIncomingData (/Users/jim/development/.../node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/Users/jim/development/.../node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:719:22)
    at Socket.emit (events.js:315:20)
    at addChunk (internal/streams/readable.js:309:12)
    at readableAddChunk (internal/streams/readable.js:284:9)
    at Socket.Readable.push (internal/streams/readable.js:223:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
    at TCP.callbackTrampoline (internal/async_hooks.js:131:14) {
  ok: 0,
  code: 148,
  codeName: 'ReadConcernMajorityNotEnabled'
})
    at ChangeStream.emit (events.js:304:17)
    at ChangeStream.<anonymous> (/Users/jim/development/.../node_modules/mongoose/lib/cursor/ChangeStream.js:41:51)
    at ChangeStream.emit (events.js:315:20)
    at processError (/Users/jim/development/.../node_modules/mongodb/lib/change_stream.js:572:38)
    at ChangeStreamCursor.<anonymous> (/Users/jim/development/.../node_modules/mongodb/lib/change_stream.js:436:5)
    at ChangeStreamCursor.emit (events.js:315:20)
    at /Users/jim/development/.../node_modules/mongodb/lib/core/cursor.js:343:16
    at /Users/jim/development/.../node_modules/mongodb/lib/core/cursor.js:736:9
    at /Users/jim/development/.../node_modules/mongodb/lib/change_stream.js:330:9
    at done (/Users/jim/development/.../node_modules/mongodb/lib/core/cursor.js:458:7)
steven130169 commented 3 years ago

same issue

jmbldwn commented 3 years ago

FYI, I was able to work around this problem, though I can't say for sure what exactly fixes this. Hopefully this will help others running into this.

Here's the code I use to establish the replica set and subsequently shut it down:

    setupModelTest() {
        const { MongoMemoryReplSet } = require('mongodb-memory-server');             // use this if mongo not on host
        // const { MongoMemoryReplSet } = require('mongodb-memory-server-core');
        let replSet;
        let connection;

        beforeAll(async () => {
            replSet = await MongoMemoryReplSet.create({
                replSet: { storageEngine: 'wiredTiger' },
            });
            await replSet.waitUntilRunning();
            const uri = await replSet.getUri();
            connection = await mongoose.connect(uri, {
                useNewUrlParser: true,
                useCreateIndex: true,
                useFindAndModify: false,
                useUnifiedTopology: true,
                poolSize: 100
            });
        });

        afterAll(async () => {
            // shut down all change streams that may be open (.stopWatch() is implemented in every model)
            await require('.../models/xxx').stopWatch();
            await require('.../models/yyy').stopWatch();

            await connection.disconnect();
            await replSet.stop();
        });
    },
steven130169 commented 3 years ago

FYI, I was able to work around this problem, though I can't say for sure what exactly fixes this. Hopefully this will help others running into this.

* I switched to using WiredTiger as the storage engine for mongodb-memory-server.

* I modified my code that uses mongodb change streams to support shutting them down (for testing purposes)

* I am using v7.0.0-beta38 of mongodb-memory-server

Here's the code I use to establish the replica set and subsequently shut it down:

    setupModelTest() {
        const { MongoMemoryReplSet } = require('mongodb-memory-server');             // use this if mongo not on host
        // const { MongoMemoryReplSet } = require('mongodb-memory-server-core');
        let replSet;
        let connection;

        beforeAll(async () => {
            replSet = await MongoMemoryReplSet.create({
                replSet: { storageEngine: 'wiredTiger' },
            });
            await replSet.waitUntilRunning();
            const uri = await replSet.getUri();
            connection = await mongoose.connect(uri, {
                useNewUrlParser: true,
                useCreateIndex: true,
                useFindAndModify: false,
                useUnifiedTopology: true,
                poolSize: 100
            });
        });

        afterAll(async () => {
            // shut down all change streams that may be open (.stopWatch() is implemented in every model)
            await require('.../models/xxx').stopWatch();
            await require('.../models/yyy').stopWatch();

            await connection.disconnect();
            await replSet.stop();
        });
    },

Thanks , I will try .