search

damianiandrea / mongodb-nats-connector

A connector that uses MongoDB's change streams to capture data changes and publishes those changes to NATS JetStream.
MIT License
48 stars 7 forks source link

Connector fails to start when using changeStreamPreAndPostImages with MongoDB versions less than 6.0 #17

Closed damianiandrea closed 7 months ago

damianiandrea commented 7 months ago

Configuring the changeStreamPreAndPostImages in the connector.yaml file while using MongoDB versions prior to 6.0 causes the connector to fail at startup:

connector  | {"time":"2024-01-28T11:15:20.04259182Z","level":"INFO","msg":"connected to mongodb","uri":"mongodb://mongo1:27017,mongo2:27017,mongo3:27017/?replicaSet=mongodb-nats-connector"}
connector  | {"time":"2024-01-28T11:15:20.053941909Z","level":"INFO","msg":"connected to nats","url":"nats://nats1:4222"}
connector  | {"time":"2024-01-28T11:15:43.242513413Z","level":"DEBUG","msg":"created mongodb collection","collName":"coll1","dbName":"test-connector"}
connector  | {"time":"2024-01-28T11:15:43.24627381Z","level":"ERROR","msg":"disconnected from nats","err":null}
connector  | {"time":"2024-01-28T11:15:43.246381976Z","level":"INFO","msg":"nats connection closed"}
connector  | 2024/01/28 11:15:43 exiting: could not enable changeStreamPreAndPostImages on mongo collection coll1: (Location40415) BSON field 'collMod.changeStreamPreAndPostImages' is an unknown field.
connector exited with code 0

This is consistent with the fact that changeStreamPreAndPostImages is a feature that was introduced in MongoDB 6.0 (more info here) but the error itself should be written to a WARN level log message, and it shouldn't prevent the connector from starting.

Furthermore, it might make sense to drop the changeStreamPreAndPostImages configuration entirely in the future. The purpose of the connector was never to provide a way to configure MongoDB. It should still enrich the published change streams with pre and post images if available, but turning on such configuration should be done directly on the database.