Closed tschaffter closed 4 years ago
I can confirm the 3.3.4-rc0 has fixed the problem š I will provide more detailed test results later.
after I changed my mongodb version in package.json to "mongodb": "3.3.4-rc0", I always get error ( this error does not happen when mongodb version is on 3.3.3):
============================================= npm ERR! path /tmp/app/node_modules/npm/node_modules/abbrev npm ERR! code ENOENT npm ERR! errno -2 npm ERR! syscall rename npm ERR! enoent ENOENT: no such file or directory, rename '/tmp/app/node_modules/npm/node_modules/abbrev' -> '/tmp/app/node_modules/npm/node_modules/.abbrev.DELETE' npm ERR! enoent This is related to npm not being able to find a file. npm ERR! enoent npm ERR! A complete log of this run can be found in: npm ERR! /home/vcap/.npm/_logs/2019-11-06T17_19_55_661Z-debug.log [31;1mERROR[0m Unable to build dependencies: exit status 254 Failed to compile droplet: Failed to run all supply scripts: exit status 14 Exit status 223
================================== my npm version is npm at 6.11.3.
Does this error relate with this new mongodb v3.3.4-rc0 build?
@zhenwan this looks like an issue downloading the abbrev
package. My guess is that there are issues with your package-lock.json
, please try deleting that file and running npm install
again.
I was able to reproduce the error and check if it was solved with the v3.3.4-rc0 driver using the following setup:
GET /api/todos
which queries a todo
MongoDB collection which contains some sample dataThe results of this test were as follows:
ā
Mongodb Node.js 3.3.2
useUnifiedTopology false
> MongoDB failover triggered
> MongoDB 'reconnect' event
> A few http calls with +/- 5000ms response times
> MongoDB failover finished
> Everything back to normal
ā Mongodb Node.js 3.3.2
useUnifiedTopology true
> MongoDB failover triggered
> MongoDB 'close' event
> Many MongoNetworkError errors
{
"name": "MongoNetworkError",
"stack": "Error: connect ECONNREFUSED <ip_redacted>:27017
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1128:14)",
"errorLabels": [
"TransientTransactionError"
]
}
> EventEmitter memory leak warning
(node:18) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 topologyDescriptionChanged listeners added to [NativeTopology]. Use emitter.setMaxListeners() to increase limit
> Many MongoTimeoutError errors
{
"name": "MongoTimeoutError",
"stack": "MongoTimeoutError: Server selection timed out after 30000 ms
at Timeout._onTimeout (/app/node_modules/mongodb/lib/core/sdam/topology.js:850:16)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timers.js:475:7)"
}
> MongoDB failover finished
> Server unable to recover MongoDB connection
> Server restarted manually
> Everything back to normal
ā ļø Mongodb Node.js 3.3.4-rc0
useUnifiedTopology true
> MongoDB failover triggered
> MongoDB 'reconnect' event expected but not seen
> A few http calls with +/- 5000ms response times
> A couple failed http calls:
{
"name": "MongoError",
"stack": "MongoError: Cache Reader No keys found for HMAC that is valid for time: { ts: Timestamp(1573064656, 1) } with id: 6756219367492419585
at Connection.<anonymous> (/app/node_modules/mongodb/lib/core/connection/pool.js:451:61)
at Connection.emit (events.js:210:5)
at processMessage (/app/node_modules/mongodb/lib/core/connection/connection.js:368:10)
at TLSSocket.<anonymous> (/app/node_modules/mongodb/lib/core/connection/connection.js:537:15)
at TLSSocket.emit (events.js:210:5)
at addChunk (_stream_readable.js:308:12)
at readableAddChunk (_stream_readable.js:289:11)
at TLSSocket.Readable.push (_stream_readable.js:223:10)
at TLSWrap.onStreamRead (internal/stream_base_commons.js:182:23)",
"ok": 0,
"errmsg": "Cache Reader No keys found for HMAC that is valid for time: { ts: Timestamp(1573064656, 1) } with id: 6756219367492419585",
"code": 211,
"codeName": "KeyNotFound"
}
> MongoDB failover finished
> Everything back to normal
So it seems to work with v3.3.4-rc0
, the server is able to recover the connection and resume serving data from MongoDB. There are still two issues though:
@mbroadst yes, deleting package-lock.json solved my missing file issue.
But I still get the error: 2019-11-06T12:52:10.35-0600 [CELL/0] OUT Container became healthy 2019-11-06T12:52:39.06-0600 [APP/PROC/WEB/0] ERR /home/vcap/app/node_modules/connect-mongo/src/index.js:135 2019-11-06T12:52:39.06-0600 [APP/PROC/WEB/0] ERR throw err 2019-11-06T12:52:39.06-0600 [APP/PROC/WEB/0] ERR ^ 2019-11-06T12:52:39.06-0600 [APP/PROC/WEB/0] ERR Error: Server selection timed out after 30000 ms 2019-11-06T12:52:39.06-0600 [APP/PROC/WEB/0] ERR at Timeout.setTimeout [as _onTimeout] (/home/vcap/app/node_modules/connect-mongo/node_modules/mongodb/lib/core/sdam/topology.js:878:9) 2019-11-06T12:52:39.06-0600 [APP/PROC/WEB/0] ERR at ontimeout (timers.js:469:11) 2019-11-06T12:52:39.06-0600 [APP/PROC/WEB/0] ERR at tryOnTimeout (timers.js:304:5) 2019-11-06T12:52:39.06-0600 [APP/PROC/WEB/0] ERR at Timer.listOnTimeout (timers.js:264:5) 2019-11-06T12:52:39.35-0600 [APP/PROC/WEB/0] OUT Exit status 1
@zhenwan it looks like it might be related to not being able to connect to your cluster on initial connect. Also your backtrace to topology.js:878
curiously does not point to a place where an error is generated, nor are you receiving the correct type MongoTimeoutError
.
Can you please open a new jira ticket so we can triage your particular case. It looks like we'll need more detail than what you've pasted here. Thank you!
@mbroadst Thanks for helping fix the issue. I just want to say, using v3.3.4-rc0 is not solving my problem. I'm still getting this message
MongoTimeoutError: Server selection timed out after 30000 ms
at Timeout.setTimeout (api/node_modules/mongodb/lib/core/sdam/topology.js:899:9)
at ontimeout (timers.js:498:11)
at tryOnTimeout (timers.js:323:5)
at Timer.listOnTimeout (timers.js:290:5)
In my case, I'm using loopback-connector-mongodb
, but I setup my package.json
to force using last version of mongodb
in resolutions
as follow
"resolutions": {
"mongodb": "^3.3.4-rc0"
}
my connection parameters as follow
"host": "127.0.0.1",
"port": 27017,
"database": "***",
"password": "***",
"user": "***",
"authSource": "***",
"useNewUrlParser": true,
"enableGeoIndexing": true
below is my environment parameters
MongoDB server version: 3.6.3 on localhost
yarn v1.19.1
node v8.16.2
Thanks in advance
update
I also tried to add useUnifiedTopology: true
to connection parameters, but got same result.
Another confirmation here, from someone using mongoose directly, that useUnifiedTopology: false
works for me.
@mbroadst thank you for the update from MongoDB's end. Excuse my ignorance, but will the aforementioned fix in mongodb v3.3.4-rc0 cascade down to mongoose at some point? Slightly uncomfortable with the idea of embracing deprecated features as a workaround in a prod environment.
@mmmmoj similar to @zhenwan's question above, your stack trace seems to indicate you are not in fact running v3.3.4-rc0. Maybe try removing your package-lock.json
? Can you please file a jira ticket so we can delve into the specifics of your case.
@bigsee (man GitHub does not like autocompleting your name!) Glad to hear it works for you, and you're welcome for the update - our community is very important to us. I'll let @vkarpov15 speak for the timing of the release, but we work closely and I'm sure he's interested in fixing this for y'all :)
@mbroadst ha! sorry for the awkward name!
it turns out I might have spoken too soon, perhaps given the random nature of the error appearing. On a cold start of my app, along with the expected deprecation warning, I now see this slightly scarier error in the logs:
Unhandled rejection: { MongoNetworkError: failed to connect to server [myserver.mlab.com:55841] on first connect [{ MongoNetworkError: connection 0 to myserver.mlab.com:55841 timed out
at Socket.<anonymous> (/var/task/node_modules/mongodb/lib/core/connection/connection.js:335:7)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:106:13)
at Socket.emit (events.js:208:7)
at Socket._onTimeout (net.js:420:8)
at ontimeout (timers.js:482:11)
at tryOnTimeout (timers.js:317:5)
at Timer.listOnTimeout (timers.js:277:5) name: 'MongoError', [Symbol(mongoErrorContextSymbol)]: {} }]
at Pool.<anonymous> (/var/task/node_modules/mongodb/lib/core/topologies/server.js:431:11)
at emitOne (events.js:116:13)
at Pool.emit (events.js:211:7)
at connect (/var/task/node_modules/mongodb/lib/core/connection/pool.js:580:14)
at callback (/var/task/node_modules/mongodb/lib/core/connection/connect.js:109:5)
at provider.auth.err (/var/task/node_modules/mongodb/lib/core/connection/connect.js:352:21)
at _authenticateSingleConnection (/var/task/node_modules/mongodb/lib/core/auth/auth_provider.js:66:11)
at sendAuthCommand (/var/task/node_modules/mongodb/lib/core/auth/scram.js:177:16)
at Connection.errorHandler (/var/task/node_modules/mongodb/lib/core/connection/connect.js:321:5)
at Object.onceWrapper (events.js:317:30)
at emitTwo (events.js:126:13)
at Connection.emit (events.js:214:7)
at Socket.<anonymous> (/var/task/node_modules/mongodb/lib/core/connection/connection.js:333:10)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:106:13)
at Socket.emit (events.js:208:7)
name: 'MongoNetworkError',
errorLabels: [ 'TransientTransactionError' ],
[Symbol(mongoErrorContextSymbol)]: {} }
It used to be MongoTimeoutError
but now it's MongoNetworkError
... ...any ideas?
@bigsee that error would indicate that you are not using the unified topology. Please enable it by passing useUnifiedTopology: true
to your MongoClient
@mbroadst that's correct, which I believe is the point of this issue. Apologies if I'm wrong on that.
When I pass useUnifiedTopology: true
the deprecation warning disappears but my app randomly crashes with the title error of this issue MongoTimeoutError: Server selection timed out after 30000 ms
.
As with @vkarpov15 and others above, I have not been able to reproduce this issue consistently. The error happens a few times a day resulting in 502 and horrible UX from time to time errors. It seems to be a new issue.
As reported above, the workaround of setting useUnifiedTopology: false
(or commenting out) prevents the error in this issue, but restores the deprecation warning and now the slightly scarier MongoNetworkError above. As an aside, this doesn't actually kill my app but does worry me that it will for a user at some point.
@bigsee the MongoNetworkError
you are experiencing is not new, and it means that the driver was unable to make an initial connection to a cluster (incidentally if some sort of race condition between your app starting and your cluster starting is causing this, then the unified topology should help here as well).
The point of this issue is that people were following the guidance to use the unified topology, and when they did they were experiencing spurious timeout errors. v3.3.4-rc0
was released to address that specific issue, so the current guidance is to use that version which should remove the deprecation notice and correct the spurious timeout issues people have encountered.
I should also say: this version will not prevent all timeout errors. That error is being caused by the driver's inability to do an initial connect to a cluster, which would indicate some sort of configuration (connection string) or network error on your part. With the unified topology this will show up as a MongoTimeoutError
with a reason
field indicating the network error that caused the timeout on initial connect.
@mbroadst I didn't mean to suggest that the MongoNetworkError
was new in my last comment, so apologies if that is how it read. It's the `MongoTimeoutError that is new. To hopefully be clearer, this has been my process:
useUnifiedTopology: true
- everything working as expectedMongoTimeoutError
matching the title of this issue, as well as the first few lines of a stack trace presented aboveuseUnifiedTopology: false
workaround suggestedMongoTimeoutError
error disappears along with the crashing of the app. however, it is replaced by the deprecation warning initially. then, later, by a additional MongoNetworkError
not seen in the application before setting useUnifiedTopology: false
Perhaps, as you suggest, this is correlation rather than causation. Just seemed like a smoking gun to me. At least the app is not crashing now. I'm just worried about leaving an option that is deprecated.
@bigsee Ah! Sorry, I misread your original comment to mean that you were confirming that the v3.3.4-rc0
bug fixes resolved your issues from v3.3.3
. Have you had a chance to try the new version?
This is my first trial on Mongodb and I'm having this error. read through the comments and changed my db to v3.3.4-rc0 and it didn't still fix the error
DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
MongoNetworkError: failed to connect to server [cluster0-shard-00-00-rhdve.mongodb.net:27017] on first connect [MongoNetworkError: connection 3 to cluster0-shard-00-00-rhdve.mongodb.net:27017 closed
at TLSSocket.
}]
at Pool.
} TypeError: Cannot read property 'find' of undefined at C:\laragon\www\Vue-express-mongodb\server\routes\api\posts.js:12:26
@Syfon01 the error you are experiencing is because of some sort of configuration error, the driver is using the legacy topology and failing to make its initial connection to a cluster after a default of 10s. Please check your connection string and network configuration and verify that you are able to connect to the cluster (perhaps with the mongo
shell). If you're still experiencing issues, please open a jira ticket so we can continue conversation there, and not confuse people looking at this GitHub issue
For people who are having this issue , if you are using Mongo Atlas Cloud Instance , Please see if you are having your IP address specified correctly in the Network Access tab.
I had the issue and it seems that since my ISP gives out New IPs everytime (DHCP) i reconnect to the network , Mongo Atlas was only white listing the previously specified IP address. Hope this helps someone.
update - doesn't work
I was able to solve this issue by changing the structure of my code, specifically the callback as follows:
mongoose.connect(DATABASE_URL, {
useNewUrlParser: true,
useUnifiedTopology: true,
}).then(() => {
console.log('Connected to DB');
app.listen({ port: PORT }, () => {
console.log(`Server running at http://localhost:${PORT}`)
});
}).catch(err => console.log(err));
mongoose.connect(DATABASE_URL, {
useNewUrlParser: true,
useUnifiedTopology: true,
}, () => {
console.log('Connected to DB');
app.listen({ port: PORT }, () => {
console.log(`Server running at http://localhost:${PORT}`);
});
}).catch(err => console.log(err));
@mtn2bay that doesn't make any sense you basically used the callback function instead of the promise, the result is still one and the same, the only difference is that in the callback function you get both the connection and the error in the callback function which you're not logging instead of relying on .then and .catch, so your function is logging connected to db and starting the app either way even if there's an error. try this instead
mongoose.connect(DATABASE_URL, {
useNewUrlParser: true,
useUnifiedTopology: true,
}, (err, connection) => {
if(err) {
console.error(err)
return
}
console.log('Connected to DB');
app.listen({ port: PORT }, () => {
console.log(`Server running at http://localhost:${PORT}`);
})
@khaledosman thank you for the correction. You are right, I was just ignoring the error rather than addressing it. I actually had an issue with my db path.
i was facing the same issue, i updated mongoose npm package from 5.6.9 to 5.7.6 and the issue vanished.
For people who are having this issue , if you are using Mongo Atlas Cloud Instance , Please see if you are having your IP address specified correctly in the Network Access tab.
I had the issue and it seems that since my ISP gives out New IPs everytime (DHCP) i reconnect to the network , Mongo Atlas was only white listing the previously specified IP address. Hope this helps someone.
solved my issue
UnhandledPromiseRejectionWarning: MongoTimeoutError: Server selection timed out after 30000 ms at Timeout.setTimeout (C:\Users\Umer MIB\Desktop\Chatbots2\mongobot\node_modules\mongodb\lib\core\sdam\topology.js:878:9) at ontimeout (timers.js:436:11) at tryOnTimeout (timers.js:300:5) at listOnTimeout (timers.js:263:5) at Timer.processTimers (timers.js:223:10) (node:4304) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:4304) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code
I do not know how that solved a problem, but I downloaded Compass and connected it to database, just copied the link. Once it got connected, the error was gone. Topology is still true in my code. Maybe it configured some options with Compass that were not right with usual mongodb installation. Now I can connect without Compass and all seems to work, just did it once in the beginning.
I got the same issue and thanks to this forum, got through by commenting useUnifiedTopology setting. However then I uncommented this setting, but now it still works just fine... quite confused...
Do you want to request a feature or report a bug?
A bug.
What is the current behavior?
After updating to Mongoose 5.7.1, the following warning appeared:
I added
useUnifiedTopology: true
to my mongoose config object as suggested by the message, even though I'm not using MongoDB replica set or sharded cluster. Here is how I configure and connect using mongoose:and here is where I used this
mongo.options
object:The warning is gone but the issue is that mongoose is now no longer able to connect:
Here is how I run MongoDB using docker during development:
If the current behavior is a bug, please provide the steps to reproduce.
See above
What is the expected behavior?
Mongoose should remove the deprecation warning and run fine when using
useUnifiedTopology: true
as mongoose suggests.What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.
Node: v10.16.2 Mongoose: 5.7.1 (latest) MongoDB: db version v4.2.0
Related issues: