error: Uncaught Error: read ECONNRESET

[mysteryjeans]

Despite access npm:mongodb in try/catch, runtime is throwing Uncaught error, whenever db drops the connection.

error: Uncaught Error: read ECONNRESET
    at node_internal_captureLargerStackTrace (ext:deno_node/internal/errors.ts:89:11)
    at node_internal_errnoException (ext:deno_node/internal/errors.ts:137:12)
    at TCP.onStreamRead [as onread] (ext:deno_node/internal/stream_base_commons.ts:205:24)
    at TCP.#read (ext:deno_node/internal_binding/stream_wrap.ts:223:18)
    at eventLoopTick (ext:core/01_core.js:166:11)

[bartlomieju]

Probably the same problem as https://github.com/denoland/rusty_v8/issues/1226. We're bisecting V8 to see what might have caused it.

[mysteryjeans]

As an C# dev, this is my first microservice to do some compute in Deno & Typescript. Do we have a work around? Although I understand db connection shouldn't be dropped / closed abruptly but need a way to handle this error and retry after sometime.

[bartlomieju]

@mysteryjeans is there a reproduction code you could share? The reconnection/retry part should be handled by the library you are using (npm:mongodb) and maybe it already does that, but there's a bug in Deno. Having a reproduction will help us fix the problem.

Also what Deno version are you using and which OS are you on?

[mysteryjeans]

I am not able to exactly reproduce same case, since our mongodb server was restarting continuously due to some resource constraint, It is now fixed; which was the root cause for connection drop.

However I am able to reproduce similar case on deno 1.33.1 and 1.33.3. It causes script to exit when I turned of wifi in the middle of long running query.

import { MongoClient } from "npm:mongodb";
import { delay } from "https://deno.land/std@0.185.0/async/mod.ts";
import secrets from "./secrets.json" assert { type: "json" };

async function main() {
  while (true) {
    try {
      const client = new MongoClient(secrets.mongodb_url);
      await client.connect();
      try {
        console.log("Connected to db..");

        const db = client.db("SomeDatabase");
        const collection = db.collection("LargeCollection");

        console.log("Querying large collection.. now disconnect wifi..");
        const results = await collection.find({ NonExistField: "Hello" }).toArray(); // Turn off wifi
        for (const result in results) {
          // Do something..
        }
      }
      finally {
        client.close();
      }
    }
    catch (err) {
      console.error("Some error occurred.. going for nap", err); 
      await delay(10000);
    }
  }
}

// Learn more at https://deno.land/manual/examples/module_metadata#concepts
if (import.meta.main) {
  await main();
}

Its successfully logging this time in catch, but it exists the script after catch and outputs connection refuse error ECONNREFUSED.

PS C:\Data\Workspace\ts-playground> deno run --allow-all .\main.ts
Connected to db..
Querying large collection..
Some error occurred.. going for nap MongoNetworkTimeoutError: connection timed out
    at connectionFailureError (file:///C:/Users/fmkhan/AppData/Local/deno/npm/registry.npmjs.org/mongodb/5.3.0/lib/cmap/connect.js:372:20)
    at Socket.<anonymous> (file:///C:/Users/fmkhan/AppData/Local/deno/npm/registry.npmjs.org/mongodb/5.3.0/lib/cmap/connect.js:293:22)
    at Object.onceWrapper (ext:deno_node/_stream.mjs:1927:32)
    at Socket.emit (ext:deno_node/_stream.mjs:1852:9)
    at Socket._onTimeout (ext:deno_node/net.ts:846:14)
    at cb (ext:deno_node/internal/timers.mjs:37:46)
    at Object.action (ext:deno_web/02_timers.js:149:11)
    at handleTimerMacrotask (ext:deno_web/02_timers.js:66:10)
    at eventLoopTick (ext:core/01_core.js:187:21) {
  connectionGeneration: 0,
  [Symbol(errorLabels)]: Set(0) {}
}
error: Uncaught Error: connect ECONNREFUSED 10.90.10.231:27017 - Local (undefined:undefined)
    at __node_internal_captureLargerStackTrace (ext:deno_node/internal/errors.ts:89:11)
    at __node_internal_exceptionWithHostPort (ext:deno_node/internal/errors.ts:213:12)
    at TCPConnectWrap._afterConnect [as oncomplete] (ext:deno_node/net.ts:168:20)
    at TCP.afterConnect (ext:deno_node/internal_binding/connection_wrap.ts:43:17)
    at ext:deno_node/internal_binding/tcp_wrap.ts:300:22
    at eventLoopTick (ext:core/01_core.js:181:11)

Deno version

PS C:\Data\Workspace\ts-playground> deno --version
deno 1.33.3 (release, x86_64-pc-windows-msvc)
v8 11.4.183.2
typescript 5.0.4

[eikooc]

I don't know if this is the same, but I experience a similar issue with mongodb after it has been running for a while (hours). Suddenly the connection is closed. I can see this from the changeStream.on('close') callback I am listening to being closed and then I get this error:

error: Uncaught Error: read ECONNRESET
    Error.captureStackTrace(err);
          ^
    at __node_internal_captureLargerStackTrace (https://deno.land/std@0.177.0/node/internal/errors.ts:113:11)
    at __node_internal_errnoException (https://deno.land/std@0.177.0/node/internal/errors.ts:194:10)
    at TCP.onStreamRead [as onread] (https://deno.land/std@0.177.0/node/internal/stream_base_commons.ts:279:20)
    at TCP.#read (https://deno.land/std@0.177.0/node/internal_binding/stream_wrap.ts:301:12)

After digging a bit deeper it could seem like what is happening is something akin to this from Node.js:

"whenever you emit an 'error' event and no one listens to it, it will throw." https://stackoverflow.com/a/11542134

[TillaTheHun0]

I took a stab at creating a minimal reproduction, and was able to reproduce in certain scenarios, here

I was only able to reproduce on an Atlas Serverless Cluster, which seems correlated, but can't say for sure without digging deeper.

[eikooc]

I've noticed that this happens every time the MongoDB cluster scales up or down so the connection is cut.

[eikooc]

I believe this has been fixed in v1.37 https://github.com/denoland/deno/pull/20314. Right now it is possible to update by running deno upgrade --canary

(At least all these errors, along with a range of others have not come since upgrading)

[TillaTheHun0]

I was still able to reproduce this issue on the latest canary build:

deno 1.36.3+329698c (canary, x86_64-unknown-linux-gnu)
v8 11.6.189.12
typescript 5.1.6

start this reproduction with a connection string to an Atlas Serverless Instance. After about 10 minutes, the next operation sent through the Mongo Driver crashes the process with the same error as before:

error: Uncaught Error: read ECONNRESET
    at __node_internal_captureLargerStackTrace (ext:deno_node/internal/errors.ts:91:9)
    at __node_internal_errnoException (ext:deno_node/internal/errors.ts:139:10)
    at TCP.onStreamRead [as onread] (ext:deno_node/internal/stream_base_commons.ts:209:20)
    at TCP.#read (ext:deno_node/internal_binding/stream_wrap.ts:277:14)
    at eventLoopTick (ext:core/01_core.js:183:11)

I don't have any way to confirm, but this seems to be something being caused on the Atlas side; it seems to be closing connections being maintained in the driver's connection pool. Perhaps the Node Driver, or even Deno could handle the ECONNRESET more gracefully, but the fact that the Atlas Serverless instance is just closing connections doesn't seem right. Maybe i'm misunderstanding, but it is as if Atlas Serverless assumes its consumer is also Serverless and so it's fine to close connections after a period of inactivity. But that isn't necessarily the case.

[bartlomieju]

@littledivy can you take a look?