search

RocketChat / Rocket.Chat

The communications platform that puts data protection first.
https://rocket.chat/
Other
40.02k stars 10.32k forks source link

Mongo 4 -5 upgrade errors - "Some indexes for collection could not be created" #31876

Open reetp opened 6 months ago

reetp commented 6 months ago

After upgrade of Mongo DB from 4 to 5 I get errors in the logs. Full log below.

This is repeatable and has occurred on 3 different instances.

Steps to reproduce:

Upgrade Mongo DB from 4 to 5

Restart rocket with docker-compose

Expected behavior:

No errors

Actual behavior:

As per logs below

Server Setup Information:

Deployment Version 6.6.2 Apps Engine Version 1.41.0 Node Version v14.21.3 Database Migration 304 (March 2, 2024 10:10 PM) MongoDB 5.0.25 / wiredTiger (oplog Enabled) Commit Details HEAD: (44b682f6b) Branch: HEAD Release 6.6.2

Client Setup Information

NA

Additional context

3 rocket.chat servers all upgraded from prior versions - one from around 0.32 or possibly earlier!

Relevant logs:

Seems the same error but different collections depening on the server

Server 1

Some indexes for collection 'rocketchat_credential_tokens' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { expireAt: 1 }, name: "expireAt_1", sparse: true, expireAfterSeconds: 0 }, existing index: { v: 2, key: { expireAt: 1 }, name: "expireAt_1", sparse: 1, expireAfterSeconds: 0 }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { expireAt: 1 }, name: "expireAt_1", sparse: true, expireAfterSeconds: 0 }, existing index: { v: 2, key: { expireAt: 1 }, name: "expireAt_1", sparse: 1, expireAfterSeconds: 0 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Some indexes for collection 'rocketchat_message' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { pinnedBy._id: 1 }, name: "pinnedBy._id_1", sparse: true }, existing index: { v: 2, key: { pinnedBy._id: 1 }, name: "pinnedBy._id_1", sparse: 1 }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { pinnedBy._id: 1 }, name: "pinnedBy._id_1", sparse: true }, existing index: { v: 2, key: { pinnedBy._id: 1 }, name: "pinnedBy._id_1", sparse: 1 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 4)

Some indexes for collection 'rocketchat_subscription' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: true }, existing index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: 1 }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: true }, existing index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: 1 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 6)

Some indexes for collection 'rocketchat_uploads' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { rid: 1 }, name: "rid_1", sparse: true }, existing index: { v: 2, key: { rid: 1 }, name: "rid_1" }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { rid: 1 }, name: "rid_1", sparse: true }, existing index: { v: 2, key: { rid: 1 }, name: "rid_1" }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 8)
(node:1) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy

{"level":40,"time":"2024-03-02T20:52:18.447Z","pid":1,"hostname":"quixote.hostdomain.com","name":"VoIPService","msg":"Voip is not enabled. Cant start the service"}
{"level":51,"time":"2024-03-02T20:52:18.498Z","pid":1,"hostname":"quixote.hostdomain.com","name":"Migrations","msg":"Not migrating, already at version 304"}
[2024-03-02T20:52:18.515Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/BROKER: Moleculer v0.14.31 is starting...
[2024-03-02T20:52:18.516Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/BROKER: Namespace: <not defined>
[2024-03-02T20:52:18.516Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/BROKER: Node ID: 8458ff1b-9f2f-44c9-afe6-9db4216b17f1
[2024-03-02T20:52:18.519Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/REGISTRY: Strategy: RoundRobinStrategy
[2024-03-02T20:52:18.520Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/REGISTRY: Discoverer: LocalDiscoverer
[2024-03-02T20:52:18.522Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/BROKER: Serializer: EJSONSerializer
[2024-03-02T20:52:18.544Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/BROKER: Validator: FastestValidator
[2024-03-02T20:52:18.548Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/BROKER: Registered 13 middleware(s).
[2024-03-02T20:52:18.551Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/BROKER: Transporter: TcpTransporter
{"level":51,"time":"2024-03-02T20:52:18.599Z","pid":1,"hostname":"quixote.hostdomain.com","name":"DatabaseWatcher","msg":"Using change streams"}
[2024-03-02T20:52:18.640Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/TRANSIT: Connecting to the transporter...
[2024-03-02T20:52:18.656Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/TRANSPORTER: TCP server is listening on port 40923
[2024-03-02T20:52:18.658Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/TRANSPORTER: UDP Discovery is disabled.
[2024-03-02T20:52:18.659Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/TRANSPORTER: TCP Transporter started.
{"level":51,"time":"2024-03-02T20:52:18.894Z","pid":1,"hostname":"quixote.hostdomain.com","name":"License","msg":"License installed","version":"3.0","hash":"1Aw4siEw"}
(node:1) [MONGODB DRIVER] Warning: cursor.count is deprecated and will be removed in the next major version, please use `collection.estimatedDocumentCount` or `collection.countDocuments` instead 
[2024-03-02T20:52:19.232Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/REGISTRY: '$node' service is registered.
[2024-03-02T20:52:19.235Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/REGISTRY: 'matrix' service is registered.
[2024-03-02T20:52:19.237Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/$NODE: Service '$node' started.
[2024-03-02T20:52:19.238Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/MATRIX: Service 'matrix' started.
[2024-03-02T20:52:19.240Z] INFO  8458ff1b-9f2f-44c9-afe6-9db4216b17f1/BROKER: ✔ ServiceBroker with 2 service(s) started successfully in 654ms.
+---------------------------------------------------------+
|                      SERVER RUNNING                     |
+---------------------------------------------------------+
|                                                         |
|  Rocket.Chat Version: 6.6.2                             |
|       NodeJS Version: 14.21.3 - x64                     |
|      MongoDB Version: 5.0.25                            |
|       MongoDB Engine: wiredTiger                        |
|             Platform: linux                             |
|         Process Port: 3000                              |
|             Site URL: https://chat.rocket.com  |
|     ReplicaSet OpLog: Enabled                           |
|          Commit Hash: 44b682f6bc                        |
|        Commit Branch: HEAD                              |
|                                                         |
+---------------------------------------------------------+
{"level":51,"time":"2024-03-02T20:52:22.043Z","pid":1,"hostname":"quixote.hostdomain.com","name":"License","msg":"License installed","version":"3.0","hash":"VX8s-M2Q"}

Server 2

Some indexes for collection 'rocketchat_room' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { open: 1 }, name: "open_1", sparse: true }, existing index: { v: 2, key: { open: 1 }, name: "open_1", sparse: 1 }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { open: 1 }, name: "open_1", sparse: true }, existing index: { v: 2, key: { open: 1 }, name: "open_1", sparse: 1 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Some indexes for collection 'rocketchat_credential_tokens' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { expireAt: 1 }, name: "expireAt_1", sparse: true, expireAfterSeconds: 0 }, existing index: { v: 2, key: { expireAt: 1 }, name: "expireAt_1", sparse: 1, expireAfterSeconds: 0 }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { expireAt: 1 }, name: "expireAt_1", sparse: true, expireAfterSeconds: 0 }, existing index: { v: 2, key: { expireAt: 1 }, name: "expireAt_1", sparse: 1, expireAfterSeconds: 0 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 4)
Some indexes for collection 'rocketchat_room' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, unique: true, key: { name: 1 }, name: "name_1", sparse: true }, existing index: { v: 2, unique: true, key: { name: 1 }, name: "name_1", sparse: 1 }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, unique: true, key: { name: 1 }, name: "name_1", sparse: true }, existing index: { v: 2, unique: true, key: { name: 1 }, name: "name_1", sparse: 1 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 6)
Some indexes for collection 'rocketchat_subscription' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: true }, existing index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: 1 }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: true }, existing index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: 1 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 8)
Some indexes for collection 'rocketchat_uploads' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { rid: 1 }, name: "rid_1", sparse: true }, existing index: { v: 2, key: { rid: 1 }, name: "rid_1" }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { rid: 1 }, name: "rid_1", sparse: true }, existing index: { v: 2, key: { rid: 1 }, name: "rid_1" }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 10)
Some indexes for collection 'rocketchat_message' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { editedAt: 1 }, name: "editedAt_1", sparse: true }, existing index: { v: 2, key: { editedAt: 1 }, name: "editedAt_1", sparse: 1 }
(node:1) UnhandledPromiseRejectionWarning: MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { editedAt: 1 }, name: "editedAt_1", sparse: true }, existing index: { v: 2, key: { editedAt: 1 }, name: "editedAt_1", sparse: 1 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 12)
(node:1) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy
{"level":40,"time":"2024-03-02T21:10:27.608Z","pid":1,"hostname":"esmith.domain.com","name":"VoIPService","msg":"Voip is not enabled. Cant start the service"}
{"level":51,"time":"2024-03-02T21:10:27.809Z","pid":1,"hostname":"esmith.domain.com","name":"Migrations","msg":"Not migrating, already at version 304"}
[2024-03-02T21:10:27.817Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/BROKER: Moleculer v0.14.31 is starting...
[2024-03-02T21:10:27.817Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/BROKER: Namespace: <not defined>
[2024-03-02T21:10:27.818Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/BROKER: Node ID: 4b14cdd4-bf58-4205-94f7-aa9ef269ef66
[2024-03-02T21:10:27.819Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/REGISTRY: Strategy: RoundRobinStrategy
[2024-03-02T21:10:27.819Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/REGISTRY: Discoverer: LocalDiscoverer
[2024-03-02T21:10:27.821Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/BROKER: Serializer: EJSONSerializer
[2024-03-02T21:10:27.828Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/BROKER: Validator: FastestValidator
[2024-03-02T21:10:27.830Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/BROKER: Registered 13 middleware(s).
[2024-03-02T21:10:27.831Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/BROKER: Transporter: TcpTransporter
{"level":30,"time":"2024-03-02T21:10:27.845Z","pid":1,"hostname":"esmith.domain.com","name":"QueueWorker","msg":"Starting queue worker"}
{"level":30,"time":"2024-03-02T21:10:27.846Z","pid":1,"hostname":"esmith.domain.com","name":"QueueWorker","msg":"Registering workers of type \"work\""}
{"level":30,"time":"2024-03-02T21:10:27.846Z","pid":1,"hostname":"esmith.domain.com","name":"QueueWorker","msg":"Registering workers of type \"workComplete\""}
{"level":51,"time":"2024-03-02T21:10:27.855Z","pid":1,"hostname":"esmith.domain.com","name":"DatabaseWatcher","msg":"Using change streams"}
{"level":30,"time":"2024-03-02T21:10:27.855Z","pid":1,"hostname":"esmith.domain.com","name":"QueueWorker","msg":"Creating indexes for queue worker"}
[2024-03-02T21:10:27.865Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/TRANSIT: Connecting to the transporter...
{"level":30,"time":"2024-03-02T21:10:27.866Z","pid":1,"hostname":"esmith.domain.com","name":"License","msg":"New Enterprise License"}
{"level":30,"time":"2024-03-02T21:10:27.870Z","pid":1,"hostname":"esmith.domain.com","name":"License","msg":"Limit validation failed","kind":"roomsPerGuest","limit":{"max":0,"behavior":"start_fair_policy"}}
[2024-03-02T21:10:27.874Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/TRANSPORTER: TCP server is listening on port 37224
[2024-03-02T21:10:27.874Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/TRANSPORTER: UDP Discovery is disabled.
[2024-03-02T21:10:27.875Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/TRANSPORTER: TCP Transporter started.
{"level":30,"time":"2024-03-02T21:10:27.894Z","pid":1,"hostname":"esmith.domain.com","name":"OmnichannelQueue","msg":"Service stopped"}
{"level":30,"time":"2024-03-02T21:10:27.958Z","pid":1,"hostname":"esmith.domain.com","name":"SAUMonitor","msg":"[aggregate] - Start Cron."}
{"level":30,"time":"2024-03-02T21:10:28.152Z","pid":1,"hostname":"esmith.domain.com","name":"EmailInbox","msg":"Clearing old email inbox registrations"}
{"level":30,"time":"2024-03-02T21:10:28.158Z","pid":1,"hostname":"esmith.domain.com","name":"Scheduler","section":"Omnichannel-Queue-Inactivity-Monitor","msg":"Service started"}
{"level":30,"time":"2024-03-02T21:10:28.176Z","pid":1,"hostname":"esmith.domain.com","name":"Scheduler","section":"AutoTransferChatScheduler","msg":"Service started"}
{"level":30,"time":"2024-03-02T21:10:28.195Z","pid":1,"hostname":"esmith.domain.com","name":"License","msg":"Limit validation failed","kind":"guestUsers","limit":{"max":0,"behavior":"start_fair_policy"}}
{"level":30,"time":"2024-03-02T21:10:28.196Z","pid":1,"hostname":"esmith.domain.com","name":"EmailInbox","msg":"Configured a total of 0 inboxes"}
{"level":30,"time":"2024-03-02T21:10:28.196Z","pid":1,"hostname":"esmith.domain.com","name":"Scheduler","section":"AutoCloseOnHoldScheduler","msg":"Service started"}
{"level":30,"time":"2024-03-02T21:10:28.251Z","pid":1,"hostname":"esmith.domain.com","name":"License","msg":"License validated","modules":[{"module":"canned-responses"},{"module":"ldap-enterprise"},{"module":"livechat-enterprise"},{"module":"voip-enterprise"},{"module":"omnichannel-mobile-enterprise"},{"module":"engagement-dashboard"},{"module":"push-privacy"},{"module":"teams-mention"},{"module":"saml-enterprise"},{"module":"oauth-enterprise"},{"module":"federation"},{"module":"videoconference-enterprise"},{"module":"message-read-receipt"},{"module":"outlook-calendar"}]}
{"level":51,"time":"2024-03-02T21:10:28.252Z","pid":1,"hostname":"esmith.domain.com","name":"License","msg":"License installed","version":"3.0","hash":"GcUNAPak"}
Some indexes for collection 'rocketchat_room' could not be created:
    An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { open: 1 }, name: "open_1", sparse: true }, existing index: { v: 2, key: { open: 1 }, name: "open_1", sparse: 1 }
=== UnHandledPromiseRejection ===
MongoServerError: An equivalent index already exists with the same name but different options. Requested index: { v: 2, key: { open: 1 }, name: "open_1", sparse: true }, existing index: { v: 2, key: { open: 1 }, name: "open_1", sparse: 1 }
    at Connection.onMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:231:30)
    at MessageStream.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:61:60)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:125:16)
    at MessageStream._write (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at addChunk (internal/streams/readable.js:293:12)
    at readableAddChunk (internal/streams/readable.js:267:9)
    at Socket.Readable.push (internal/streams/readable.js:206:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
    at TCP.callbackTrampoline (internal/async_hooks.js:130:17) {
  ok: 0,
  code: 85,
  codeName: 'IndexOptionsConflict',
  '$clusterTime': {
    clusterTime: new Timestamp({ t: 1709413827, i: 3 }),
    signature: {
      hash: new Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
      keyId: 0
    }
  },
  operationTime: new Timestamp({ t: 1709413827, i: 3 }),
  [Symbol(errorLabels)]: Set(0) {}
}
---------------------------------
Errors like this can cause oplog processing errors.
Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
Future node.js versions will automatically exit the process
=================================
[2024-03-02T21:10:28.400Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/REGISTRY: '$node' service is registered.
[2024-03-02T21:10:28.402Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/REGISTRY: 'matrix' service is registered.
[2024-03-02T21:10:28.402Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/$NODE: Service '$node' started.
[2024-03-02T21:10:28.402Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/MATRIX: Service 'matrix' started.
[2024-03-02T21:10:28.403Z] INFO  4b14cdd4-bf58-4205-94f7-aa9ef269ef66/BROKER: ✔ ServiceBroker with 2 service(s) started successfully in 556ms.
{"level":30,"time":"2024-03-02T21:10:28.545Z","pid":1,"hostname":"esmith.domain.com","name":"LDAP","msg":"Enabling LDAP Background Sync","jobName":"LDAP_Sync"}
(node:1) [MONGODB DRIVER] Warning: cursor.count is deprecated and will be removed in the next major version, please use `collection.estimatedDocumentCount` or `collection.countDocuments` instead 
{"level":30,"time":"2024-03-02T21:10:28.551Z","pid":1,"hostname":"esmith.domain.com","name":"LDAP","msg":"Enabling LDAP Background Sync","jobName":"LDAP_Sync"}
{"level":30,"time":"2024-03-02T21:10:28.930Z","pid":1,"hostname":"esmith.domain.com","name":"CROWD","msg":"Disabling CROWD Background Sync"}
{"level":30,"time":"2024-03-02T21:10:28.931Z","pid":1,"hostname":"esmith.domain.com","name":"System","msg":"Using GridFS for custom sounds storage"}
{"level":30,"time":"2024-03-02T21:10:28.932Z","pid":1,"hostname":"esmith.domain.com","name":"System","msg":"Using GridFS for custom emoji storage"}
+----------------------------------------------------+
|                   SERVER RUNNING                   |
+----------------------------------------------------+
|                                                    |
|  Rocket.Chat Version: 6.6.2                        |
|       NodeJS Version: 14.21.3 - x64                |
|      MongoDB Version: 5.0.25                       |
|       MongoDB Engine: wiredTiger                   |
|             Platform: linux                        |
|         Process Port: 3000                         |
|             Site URL: https://chat.domain.com |
|     ReplicaSet OpLog: Enabled                      |
|          Commit Hash: 44b682f6bc                   |
|        Commit Branch: HEAD                         |
|                                                    |
+----------------------------------------------------+
{"level":30,"time":"2024-03-02T21:10:29.127Z","pid":1,"hostname":"esmith.domain.com","name":"Rocket.Chat Apps","msg":"\"apps.statusUpdate\" event received for app \"3b387ba9-f57c-44c6-9810-8c0256abd64c\", but the status is the same"}
Visbhavesh commented 5 months ago

I wanted to work in thus issue please Assign this issue to me.

reetp commented 3 months ago

I wanted to work in thus issue please Assign this issue to me.

No one assigns issues here.

https://developer.rocket.chat/contribute-to-rocket.chat/modes-of-contribution/participate-in-rocket.chat-development

The answer is to delete the old indexes and let Rocket re-create them.

Possibly needs manual intervention in the DB.

reetp commented 3 months ago

So I went through the DB using Studio3T and dropped each index.

Some new ones re-appeared =on restart so I stopped, removed and started again until they were all cleared. Seems quite happy now.

Not sure how this occurred, and why Rocket can't heal itself?

sampaiodiego commented 3 months ago

this is weird.. maybe a bug on MongoDB? a few errors from your logs are like this one:

  Requested index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: true }, 
   existing index: { v: 2, key: { desktopNotifications: 1 }, name: "desktopNotifications_1", sparse: 1 }

as we can see the index is the exact the same, but MongoDB might have converted during migration to sparse: 1 and since the code didn't change we know we've always been providing sparse: true.. so it is either a bug during MongoDB migration or during the validation I'd say..

reetp commented 3 months ago

Hello Mr Sampiao ;-)

Hope you are well!!

Yup. Very strange.

Seemed to occur around when I upgraded to 5.x

I have a native mongo from rpms and rocket in docker.

I deleted the logged index issues and then new ones appeared.

Work docker container took about 10 rocket restarts to clear them all.

I have some old backups if you want to look further. You can get me on open at reetp :-)