Different behavior between node and bun when using the Neo4J driver, maybe something around TCP, works with node

nicholasoxford commented 4 months ago

What version of Bun is running?

1.1.20+ae1948925

What platform is your computer?

Darwin 23.4.0 arm64 arm

What steps can reproduce the bug?

Only happens on Neo4J clusters, which require a paid plan. Using bolt+s connecting works, but it isnt feasible to use this type of connection when using Neo4J cluster. When you switch over to using neo4j+s the connection just hangs for 60 seconds.

Related issue #neo4j/neo4j-javascript-driver/issues/1207

Script to reproduce, if you need paid connection variables you can dm me on twitter @ApolloToday

Run with node and bun

index.mjs

import { auth, driver } from "neo4j-driver";

export const GRAPH_DB_URL = process.env.GRAPH_DB_URL || "bolt://localhost:7687";

export const GRAPH_DB_USERNAME = process.env.GRAPH_DB_USERNAME || "neo4j";

export const GRAPH_DB_PASSWORD = process.env.GRAPH_DB_PASSWORD || "neo4j";
console.log({
  GRAPH_DB_URL,
  GRAPH_DB_USERNAME,
  GRAPH_DB_PASSWORD,
});
const graphDb = driver(
  GRAPH_DB_URL,
  auth.basic(GRAPH_DB_USERNAME, GRAPH_DB_PASSWORD),
  {
    maxConnectionLifetime: 50 * 60 * 1000,

    logging: {
      level: "debug",
      logger: (level, message) => {
        console[level].call(console, `${level.toUpperCase()} ${message}`);
      },
    },
  }
);

async function main() {
  const session = graphDb.session({
    database: "neo4j",
    defaultAccessMode: "WRITE",
  });
  console.log("about to run");
  const result = await session.run("MATCH (n) RETURN n");
  console.log({ result });
}

await main();

What is the expected behavior?

Expected output would look something like this

DEBUG Connection [0][] created towards <DID>.databases.neo4j.io:7687(104.197.20.211)
DEBUG Connection [0][] C: HELLO {"user_agent":"neo4j-javascript/5.22.0","bolt_agent":{"product":"neo4j-javascript/5.22.0","platform":"darwin 23.4.0; arm64","language_details":"Node/20.12.2 (v8 11.3.244.8-node.19)"},"routing":{"address":"9b0ce010.databases.neo4j.io:7687"}}
DEBUG Connection [0][] C: LOGON { ... }
DEBUG Connection [0][] S: SUCCESS {"signature":112,"fields":[{"server":"Neo4j/5.22-aura","connection_id":"bolt-57291","hints":{"connection.recv_timeout_seconds":{"low":60,"high":0},"telemetry.enabled":true}}]}
DEBUG Connection [0][] S: SUCCESS {"signature":112,"fields":[{}]}
DEBUG Connection [0][bolt-57291] created for the pool <DID>.databases.neo4j.io:7687
DEBUG Connection [0][bolt-57291] acquired from the pool<DID>.databases.neo4j.io:7687

Followed by a low of another logs and console log of the records

What do you see instead?

INFO Routing driver 0 created for server address 9b0ce010.databases.neo4j.io:7687
about to run
INFO Routing table is stale for database: "neo4j" and access mode: "WRITE": RoutingTable[database=neo4j, expirationTime=0, currentTime=1721828747606, routers=[], readers=[], writers=[]]

WARN ConnectionHolder got an error while releasing the connection. Error Neo4jError: Connection acquisition timed out in 60000 ms. Pool status: Active conn count = 0, Idle conn count = 0.. Stacktrace: Neo4jError: Connection acquisition timed out in 60000 ms. Pool status: Active conn count = 0, Idle conn count = 0.
    at new Neo4jError (/Users/noxford/Documents/protocol-internal/node_modules/neo4j-driver-core/lib/error.js:75:16)
    at newError (/Users/noxford/Documents/protocol-internal/node_modules/neo4j-driver-core/lib/error.js:111:16)
    at <anonymous> (/Users/noxford/Documents/protocol-internal/node_modules/neo4j-driver-bolt-connection/lib/pool/pool.js:134:60)
619 |     };
620 |     return Result;
621 | }());
622 | _a = Symbol.toStringTag;
623 | function captureStacktrace() {
624 |     var error = new Error('');
                          ^
Neo4jError: Connection acquisition timed out in 60000 ms. Pool status: Active conn count = 0, Idle conn count = 0.
 code: "N/A"

      at captureStacktrace (/Users/noxford/Documents/protocol-internal/node_modules/neo4j-driver-core/lib/result.js:624:21)
      at new Result (/Users/noxford/Documents/protocol-internal/node_modules/neo4j-driver-core/lib/result.js:51:53)
      at /Users/noxford/Documents/protocol-internal/node_modules/neo4j-driver-core/lib/session.js:191:43
      at /Users/noxford/Documents/protocol-internal/node_modules/neo4j-driver-core/lib/session.js:141:13
      at /Users/noxford/Documents/protocol-internal/test.mjs:37:32
      at main (/Users/noxford/Documents/protocol-internal/test.mjs:31:20)
      at module code (/Users/noxford/Documents/protocol-internal/test.mjs:41:7)

Additional information

This is production blocking for me!

nicholasoxford commented 3 months ago

Hey @Jarred-Sumner this is very much production blocking for me.

Happy to DM you environment variables, my username is @ApolloToday on X

inc16sec commented 3 months ago

nicholasoxford commented 3 months ago

@inc16sec seems to be the same problem! I might try to figure where in the neo4j library its going wrong

nicholasoxford commented 3 months ago

After some debugging, I believe where Node and Bun start behaving differently is the .onMessage on the function handshake in the bolt-connection subrepo

https://github.com/neo4j/neo4j-javascript-driver/blob/5.0/packages/bolt-connection/src/bolt/handshake.js#L114

Maybe events arent being emitted correctly? cc @Jarred-Sumner

With node, my logs show that channel.write(newHandshakeBuffer()) gets called then channel.onMessage gets triggered. Which makes sense.

inc16sec commented 3 months ago

@Jarred-Sumner Any idea what's going on with onMessage??

nicholasoxford commented 3 months ago

After some more research, it seems more related to the file node-channel.js https://github.com/neo4j/neo4j-javascript-driver/blob/5.0/packages/bolt-connection/src/channel/node/node-channel.js#L161C20-L161C23, which I see is using the net package I also see that Bun states it doesn't have full compatibility with node net

I am still digging but the hold up is probably related more this node-channel file than specifically that onMessage line, I am still crawling through

LUK3D commented 1 week ago

After some more research, it seems more related to the file node-channel.js https://github.com/neo4j/neo4j-javascript-driver/blob/5.0/packages/bolt-connection/src/channel/node/node-channel.js#L161C20-L161C23, which I see is using the net package I also see that Bun states it doesn't have full compatibility with node net

I am still digging but the hold up is probably related more this node-channel file than specifically that onMessage line, I am still crawling through

Any update on this? I'm also facing the same issue

oven-sh / bun