Connection ignoring the `compressors` option

M-Gonzalo commented 2 weeks ago

Prerequisites

[X] I have written a descriptive issue title
[X] I have searched existing issues to ensure the bug has not already been reported

Mongoose version

8.6.1

Node.js version

latest

MongoDB server version

6.8.0

Typescript version (if applicable)

No response

Description

I tried configuring my connection to use network compression with the following option:

{
  compressors: ['zstd', 'snappy', 'zlib']
}

and then verifying it is actually applied using

mongoose.connection.db!.command({ hello: 1 })

but there is no compressors section at all. These are the top level properties of the output (not showing the whole thing for security reasons):

[
  "$clusterTime", "connectionId", "electionId", "hosts", "isWritablePrimary", "lastWrite", "localTime",
  "logicalSessionTimeoutMinutes", "maxBsonObjectSize", "maxMessageSizeBytes", "maxWireVersion",
  "maxWriteBatchSize", "me", "minWireVersion", "ok", "operationTime", "primary", "readOnly", "secondary",
  "setName", "setVersion", "tags", "topologyVersion"
]

Steps to Reproduce

As shown in description

Expected Behavior

I should be seeing the actual compression algorithm negotiated between de driver and the server

M-Gonzalo commented 2 weeks ago

I've seen this ranging from node 16 and mongoose 5, to the latest bun canary and mongoose@latest

vkarpov15 commented 2 weeks ago

I'm unable to repro. RUnning the following script:

const mongoose = require('mongoose');

mongoose.connect('mongodb://127.0.0.1:27017/mongoose_test?replicaSet=rs', { compressors: ['zstd'] }).then(() => {
  return mongoose.connection.db.command({ hello: 1 })
}).then(res => console.log(res, Object.keys(res)));

and I get the following output, which includes the compression option:

{
  topologyVersion: { processId: new ObjectId('66e2f9b7243537af4ef88862'), counter: 6 },
  hosts: [ 'localhost:27017', 'localhost:27018', 'localhost:27019' ],
  setName: 'rs',
  setVersion: 1,
  isWritablePrimary: true,
  secondary: false,
  primary: 'localhost:27017',
  me: 'localhost:27017',
  electionId: new ObjectId('7fffffff0000000000000001'),
  lastWrite: {
    opTime: { ts: new Timestamp({ t: 1726151108, i: 7 }), t: 1 },
    lastWriteDate: 2024-09-12T14:25:08.000Z,
    majorityOpTime: { ts: new Timestamp({ t: 1726151108, i: 7 }), t: 1 },
    majorityWriteDate: 2024-09-12T14:25:08.000Z
  },
  maxBsonObjectSize: 16777216,
  maxMessageSizeBytes: 48000000,
  maxWriteBatchSize: 100000,
  localTime: 2024-09-12T14:25:10.883Z,
  logicalSessionTimeoutMinutes: 30,
  connectionId: 41,
  minWireVersion: 0,
  maxWireVersion: 13,
  readOnly: false,
  compression: [ 'zstd' ],
  ok: 1,
  '$clusterTime': {
    clusterTime: new Timestamp({ t: 1726151108, i: 7 }),
    signature: {
      hash: Binary.createFromBase64('AAAAAAAAAAAAAAAAAAAAAAAAAAA=', 0),
      keyId: 0
    }
  },
  operationTime: new Timestamp({ t: 1726151108, i: 7 })
} [
  'topologyVersion',
  'hosts',
  'setName',
  'setVersion',
  'isWritablePrimary',
  'secondary',
  'primary',
  'me',
  'electionId',
  'lastWrite',
  'maxBsonObjectSize',
  'maxMessageSizeBytes',
  'maxWriteBatchSize',
  'localTime',
  'logicalSessionTimeoutMinutes',
  'connectionId',
  'minWireVersion',
  'maxWireVersion',
  'readOnly',
  'compression',
  'ok',
  '$clusterTime',
  'operationTime'
]

Can you please clarify what you mean by "configuring my connection to use network compression" with a code sample?

M-Gonzalo commented 2 weeks ago

Can you please clarify what you mean by "configuring my connection to use network compression" with a code sample?

const options = mongoURI.includes('localhost') ?
    {} : { compressors: ['zstd', 'snappy', 'zlib'] } as ConnectOptions

export default (startTime: number) => mongoose.connect(mongoURI, options)
    .then(async mongoose => {
        const serverStatus = await mongoose.connection.db!.command({ hello: 1 })
        console.log(Object.keys(serverStatus).includes('compressors') ? '🟢 Compression enabled' : '🔴 Compression disabled')
    })
    .then(() => console.log(`🟢 Connected to MongoDB in ${(performance.now() - startTime).toFixed(2)}ms`))
    .catch((err) => { console.error('❌ MongoDB connection error:', err, '\nQuiting now... 💀'); process.exit(2) })

Result:

M-Gonzalo commented 2 weeks ago

OK, I can see the problem is in the URL, actually... I'll try to post it while anonymizing it

mongodb+srv://[REDACTED]:[REDACTED]@[REDACTED].mongodb.net/DATABASE-NAME?retryWrites=true&w=majority

Maybe that URL is malformed or somehow impedes the compressors config? Because in localhost compression works just fine

vkarpov15 commented 2 weeks ago

Is it possible that your mongoURI includes localhost? I presume you've tried printing out mongoURI?

M-Gonzalo commented 2 weeks ago

Is it possible that your mongoURI includes localhost? I presume you've tried printing out mongoURI?

No, it's a Mongo Atlas URL. No "localhost" there

vkarpov15 commented 2 weeks ago

Have you tried setting compressors option in your query string instead of as connection options? See MongoDB docs. You might have better luck with that, although both query string and connect options work on my end with Atlas.

M-Gonzalo commented 2 weeks ago

Have you tried setting compressors option in your query string instead of as connection options? See MongoDB docs. You might have better luck with that, although both query string and connect options work on my end with Atlas.

Yes I have. Same results.

vkarpov15 commented 1 week ago

What permissions and roles does the database user have on your Atlas DB? Maybe for some reason compression doesn't get sent back if you don't have admin permissions?

M-Gonzalo commented 1 week ago

What permissions and roles does the database user have on your Atlas DB? Maybe for some reason compression doesn't get sent back if you don't have admin permissions?

It's the driver actually who rejects the compressors option. It's like it doesn't know it, but it clearly should.

vkarpov15 commented 1 week ago

Ah ok I see what the issue might be. You're checking for the compressors option, but the runCommand result includes the compression result as a compression property, and 'compressors' !== 'compression'. Can you try that?

M-Gonzalo commented 1 week ago

Ah ok I see what the issue might be. You're checking for the compressors option, but the runCommand result includes the compression result as a compression property, and 'compressors' !== 'compression'. Can you try that?

I did, it was one of the first things I reported:

[
  "$clusterTime", "connectionId", "electionId", "hosts", "isWritablePrimary", "lastWrite", "localTime",
  "logicalSessionTimeoutMinutes", "maxBsonObjectSize", "maxMessageSizeBytes", "maxWireVersion",
  "maxWriteBatchSize", "me", "minWireVersion", "ok", "operationTime", "primary", "readOnly", "secondary",
  "setName", "setVersion", "tags", "topologyVersion"
]

These are all first level the properties I get when I connect. With my local MongoDB, I do receive a "compression" property, oddly enough:

[
  "compression", "isWritablePrimary", "topologyVersion", "maxBsonObjectSize", "maxMessageSizeBytes",
  "maxWriteBatchSize", "localTime", "logicalSessionTimeoutMinutes", "connectionId", "minWireVersion",
  "maxWireVersion", "readOnly", "ok"
]

This is why I suspect the URI must be involved somehow

vkarpov15 commented 2 days ago

Can you try running

const options = mongoURI.includes('localhost') ?
    {} : { compressors: ['zstd', 'snappy', 'zlib'] } as ConnectOptions

console.log('MongoURI:', mongoURI);

export default (startTime: number) => mongoose.connect(mongoURI, options)
    .then(async mongoose => {
        const serverStatus = await mongoose.connection.db!.command({ hello: 1 })
        console.log(Object.keys(serverStatus).includes('compression') ? '🟢 Compression enabled' : '🔴 Compression disabled')
    })
    .then(() => console.log(`🟢 Connected to MongoDB in ${(performance.now() - startTime).toFixed(2)}ms`))
    .catch((err) => { console.error('❌ MongoDB connection error:', err, '\nQuiting now... 💀'); process.exit(2) })

and paste the output with mongouri omitted?

M-Gonzalo commented 2 days ago

Can you try running

const options = mongoURI.includes('localhost') ?
    {} : { compressors: ['zstd', 'snappy', 'zlib'] } as ConnectOptions

console.log('MongoURI:', mongoURI);

export default (startTime: number) => mongoose.connect(mongoURI, options)
    .then(async mongoose => {
        const serverStatus = await mongoose.connection.db!.command({ hello: 1 })
        console.log(Object.keys(serverStatus).includes('compression') ? '🟢 Compression enabled' : '🔴 Compression disabled')
    })
    .then(() => console.log(`🟢 Connected to MongoDB in ${(performance.now() - startTime).toFixed(2)}ms`))
    .catch((err) => { console.error('❌ MongoDB connection error:', err, '\nQuiting now... 💀'); process.exit(2) })

and paste the output with mongouri omitted?

Sure.

MongoURI: mongodb+srv://######
🟢 Connected to Redis in 17.13ms
🔴 Compression disabled
🟢 Connected to MongoDB in 2878.98ms

vkarpov15 commented 2 days ago

This is very strange. Maybe try:

const options = mongoURI.includes('localhost') ?
    {} : { compressors: ['zstd', 'snappy', 'zlib'] } as ConnectOptions

console.log('Options:', options);

export default (startTime: number) => mongoose.connect(mongoURI, options)
    .then(async mongoose => {
        const serverStatus = await mongoose.connection.db!.command({ hello: 1 })
        console.log(Object.keys(serverStatus));
        console.log(mongoose.connection.client.topology.s.description);
    })
    .then(() => console.log(`🟢 Connected to MongoDB in ${(performance.now() - startTime).toFixed(2)}ms`))
    .catch((err) => { console.error('❌ MongoDB connection error:', err, '\nQuiting now... 💀'); process.exit(2) })

M-Gonzalo commented 2 days ago

This is very strange. Maybe try:

const options = mongoURI.includes('localhost') ?
    {} : { compressors: ['zstd', 'snappy', 'zlib'] } as ConnectOptions

console.log('Options:', options);

export default (startTime: number) => mongoose.connect(mongoURI, options)
    .then(async mongoose => {
        const serverStatus = await mongoose.connection.db!.command({ hello: 1 })
        console.log(Object.keys(serverStatus));
        console.log(mongoose.connection.client.topology.s.description);
    })
    .then(() => console.log(`🟢 Connected to MongoDB in ${(performance.now() - startTime).toFixed(2)}ms`))
    .catch((err) => { console.error('❌ MongoDB connection error:', err, '\nQuiting now... 💀'); process.exit(2) })

I got a bunch of stuff, some probably sensitive. What are you needing from that, exactly, so I can copy it??

Automattic / mongoose