search

reactioncommerce / reaction

Mailchimp Open Commerce is an API-first, headless commerce platform built using Node.js, React, GraphQL. Deployed via Docker and Kubernetes.
https://mailchimp.com/developer/open-commerce/
GNU General Public License v3.0
12.33k stars 2.17k forks source link

MongoError The $changeStream stage is only supported on replica sets #5743

Closed sagargupta closed 4 years ago

sagargupta commented 4 years ago

Getting an error every time I'm run the docker image version release 2.7.0 and release 2.8.0 using external MongoDB installation:

Steps to Reproduce

Please provide starting context, i.e. logged in as a user, configure a particular payment method.

  1. Pull the image from docker hub (release 2.7.0 or 2.8.0),
  2. Run/launch the docker image the first time - Use external MongoDB by providing MONGO_URL
  3. It ends abruptly with a MongoError error - MongoError: The $changeStream stage is only supported on replica sets Have tried with multiple docker images versions See docker log below:

Log from docker:

Note: you are using a pure-JavaScript implementation of bcrypt. While this implementation will work correctly, it is known to be approximately three times slower than the native implementation. In order to use the native implementation instead, run

meteor npm install --save bcrypt

in the root directory of your application. 10:32:30.298Z WARN Reaction: No shop matching domain 'null'. Defaulting to Primary Shop. 10:32:30.408Z WARN Reaction: No shop matching domain 'null'. Defaulting to Primary Shop. 10:32:32.753Z INFO Reaction: Running pre-startup function "preStartup" for plugin "reaction-inventory"... 10:32:32.754Z INFO Reaction: pre-startup function "preStartup" for plugin "reaction-inventory" finished in 1ms 10:32:32.755Z INFO Reaction: Running pre-startup function "preStartup" for plugin "reaction-orders"... 10:32:32.756Z INFO Reaction: pre-startup function "preStartup" for plugin "reaction-orders" finished in 1ms 10:32:32.756Z INFO Reaction: Running pre-startup function "preStartup" for plugin "reaction-taxes"... 10:32:32.760Z INFO Reaction: pre-startup function "preStartup" for plugin "reaction-taxes" finished in 4ms 10:32:32.760Z INFO Reaction: Running pre-startup function "preStartup" for plugin "reaction-pricing"... 10:32:32.762Z INFO Reaction: pre-startup function "preStartup" for plugin "reaction-pricing" finished in 2ms 10:32:32.762Z INFO Reaction: Running startup function "startup" for plugin "reaction-accounts"... 10:32:49.049Z INFO Reaction: Startup function "startup" for plugin "reaction-accounts" finished in 16287ms 10:32:49.049Z INFO Reaction: Running startup function "startup" for plugin "reaction-cart"... 10:32:49.049Z INFO Reaction: Startup function "startup" for plugin "reaction-cart" finished in 0ms 10:32:49.049Z INFO Reaction: Running startup function "startup" for plugin "reaction-catalog"... 10:32:49.050Z INFO Reaction: Startup function "startup" for plugin "reaction-catalog" finished in 1ms 10:32:49.050Z INFO Reaction: Running startup function "startup" for plugin "reaction-email"... 10:32:49.051Z INFO Reaction: Startup function "startup" for plugin "reaction-email" finished in 1ms 10:32:49.051Z INFO Reaction: Running startup function "startup" for plugin "reaction-file-collections"... 10:32:49.096Z INFO Reaction: Startup function "startup" for plugin "reaction-file-collections" finished in 45ms 10:32:49.096Z INFO Reaction: Running startup function "startup" for plugin "reaction-i18n"... 10:32:49.097Z INFO Reaction: Startup function "startup" for plugin "reaction-i18n" finished in 0ms 10:32:49.097Z INFO Reaction: Running startup function "startup" for plugin "reaction-inventory"... 10:32:49.098Z INFO Reaction: Startup function "startup" for plugin "reaction-inventory" finished in 0ms 10:32:49.098Z INFO Reaction: Running startup function "shopCreateListener" for plugin "reaction-navigation"... 10:32:49.099Z INFO Reaction: Startup function "shopCreateListener" for plugin "reaction-navigation" finished in 0ms 10:32:49.099Z INFO Reaction: Running startup function "startup" for plugin "reaction-orders"... 10:32:49.099Z INFO Reaction: Startup function "startup" for plugin "reaction-orders" finished in 0ms 10:32:49.099Z INFO Reaction: Running startup function "startup" for plugin "discount-codes"... 10:32:49.100Z INFO Reaction: Startup function "startup" for plugin "discount-codes" finished in 1ms 10:32:49.100Z INFO Reaction: Running startup function "startup" for plugin "reaction-email-smtp"... 10:32:49.100Z INFO Reaction: Startup function "startup" for plugin "reaction-email-smtp" finished in 0ms 10:32:49.101Z INFO Reaction: Running startup function "startup" for plugin "reaction-email-templates"... 10:32:49.101Z INFO Reaction: Startup function "startup" for plugin "reaction-email-templates" finished in 0ms 10:32:49.101Z INFO Reaction: Running startup function "startup" for plugin "reaction-job-queue"... 10:32:49.119Z INFO Reaction: Startup function "startup" for plugin "reaction-job-queue" finished in 18ms 10:32:49.119Z INFO Reaction: Running startup function "startup" for plugin "reaction-notification"... 10:32:49.120Z INFO Reaction: Startup function "startup" for plugin "reaction-notification" finished in 1ms 10:32:49.120Z INFO Reaction: Running startup function "startup" for plugin "example-paymentmethod"... 10:32:49.120Z INFO Reaction: Startup function "startup" for plugin "example-paymentmethod" finished in 0ms 10:32:49.120Z INFO Reaction: Running startup function "startup" for plugin "reaction-simple-inventory"... 10:32:49.121Z INFO Reaction: Startup function "startup" for plugin "reaction-simple-inventory" finished in 1ms 10:32:49.121Z INFO Reaction: Running startup function "startup" for plugin "reaction-pricing"...

/app/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/utils.js:132 throw err; ^ MongoError: The $changeStream stage is only supported on replica sets at queryCallback (/app/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/cursor.js:248:25) at /app/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:532:18 at _combinedTickCallback (internal/process/next_tick.js:131:7) at process._tickDomainCallback (internal/process/next_tick.js:218:9)

Possible Solution

fix the issue so it can work with external MongoDB installations in the docker image

Versions

node version is the same as one running within the docker image release 2.7.0 and 2.8.0

spencern commented 4 years ago

@sagargupta Can you share details about your MongoDB setup?

sagargupta commented 4 years ago

Official docker image: Mongo listed at https://hub.docker.com/_/mongo/

mongo:4.2.0

With docker environment variables set as:

PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin GOSU_VERSION = 1.11 JSYAML_VERSION = 3.13.0 GPG_KEYS = E162F504A20CDF15827F718D4B7C549A058F8B6B MONGO_PACKAGE = mongodb-org MONGO_REPO = repo.mongodb.org MONGO_MAJOR = 4.2 MONGO_VERSION = 4.2.0 wiredTigerCacheSizeGB = 4 MONGO_INITDB_ROOT_USERNAME = someadmin MONGO_INITDB_ROOT_PASSWORD = Someadmin243332332344

loan-laux commented 4 years ago

@sagargupta You need to run Reaction Commerce on a MongoDB replica set. A single MongoDB instance with no replication won't work.

Generally speaking, running MongoDB outside of a RS or in a RS with less than 3 instances is considered bad practice and unsafe. Have a look at the MongoDB docs to understand why and how to set up your own replica set. https://docs.mongodb.com/manual/tutorial/deploy-replica-set/

If you don't want to deal with maintaining your own replica set (which can indeed be quite a hassle if you aren't familiar enough with MongoDB), I would suggest using MongoDB Atlas. You can get a 3-server replica set for free (M0), which would be a good place to start IMO.

sagargupta commented 4 years ago

@loan-laux True that's how we run most of our production apps, however, I'm just test driving RC in a development box. Any means to evaluate this without running replica that too within the limits of local network?

loan-laux commented 4 years ago

@sagargupta For development, I'd suggest using reaction-platform, which will spin up its own MongoDB replica set all by itself.

spencern commented 4 years ago

@sagargupta I'm going to close this as I believe this is expected behavior. Reaction does support external MongoDB installations, but does require a replica set as Loan explains so well above. As mentioned, the Reaction Platform project setups up a MongoDB replica set suitable for development.

TomMD commented 4 years ago

Closing without a straight-forward path is unfortunate. An application is using MongoDB. The developer wishes to test locally but has no solution to do so, ideally doing something like:

docker run --rm -p 27017:27017 mongo:4.2-fake-rs
./testApplication

Such local tests work for mongo up until you need a feature that only functions in the presence of replica sets. At that point there doesn't seem to be as easy a test solution, indeed it would be

quite a hassle if you aren't familiar enough with MongoDB

So how can we move forward? Can we have a mongodb flag that allows change sets even without replica sets? A docker instance that runs many instances of mongo with supervisord and sets up replica sets for the developer?

janus-reith commented 4 years ago

@TomMD What part exactly is missing for you in order to have a straight-forward path to do local tests? I think there is some confusion here that needs to be cleared up regarding the complexity of a mongodb replica set for local development:

Note that a replica set can also be initialized with a single instance of mongodb. It doesn't even involve manual configuration except for the commands passed to the mongo instance, like done here for the default reaction dev environment: https://github.com/reactioncommerce/reaction/blob/8b3d66d758c8fe0e2ba1df1958767587ddb7a046/docker-compose.yml#L29

The reaction-platform script @loan-laux and @spencern mentioned would spin up the containers described in the docker-compose file I linked, and that's basically everything to get the api and a mongodb replica set running for local development.

If you opted out of the reaction-platform script for some reason, just make sure to start mongodb with these args.

TomMD commented 4 years ago

There is indeed some confusion, such as my thinking I was reading an issue on the MongoDB repo. Thanks for the information-rich reply @janus-reith .