Fix encoding password in MongoDB connection URI

[thundermiracle]

Thanks for your great library!

I'm facing a bug(connection url parsing error) caused by mongodb@v3 when password contains ? which generated by AWS.

Would you please consider publish the latest cli with mongodb@v5?

[pkosiec]

Hey @thundermiracle, sure thing 👍 I'll try to do that on Monday, will keep you posted. Cheers!

[thundermiracle]

caused by mongodb@v3 when password contains /

Sorry. ? is the problem.

The problem regex's source code is here: https://github.com/mongodb/node-mongodb-native/blob/v3.7.4/lib/core/uri_parser.js#L13

[hayashi-ay]

Hi @pkosiec,

In addition to @thundermiracle comments, I can reproduce the issue with the following code. I will check whether upgrading the mongodb package to the latest version solves the problem.

const qs = require('querystring');

const HOSTS_RX = /(mongodb(?:\+srv|)):\/\/(?: (?:[^:]*) (?: : ([^@]*) )? @ )?([^/?]*)(?:\/|)(.*)/;

let password = "contain?character"
let uri = `mongodb://user:${password}@hostname:27017/db?authSource=admin`

const cap = uri.match(HOSTS_RX);
const dbAndQuery = cap[4].split('?');
const db = dbAndQuery.length > 0 ? dbAndQuery[0] : null;
const query = dbAndQuery.length > 1 ? dbAndQuery[1] : null;
let parsedQueryString = qs.parse(query);
for (const key in parsedQueryString) {
    let value = parsedQueryString[key]
    if (value === '' || value == null) {
        console.error(`${key} is empty or null`)
    } else {
        console.log(`[${key}, ${value}]`);
    }
}

[pkosiec]

Hello folks, Thank you for your comments. Good news - 3.8.0-alpha.0 is ready to test! It includes the latest official MongoDB driver (v6). Could you please try it and check if you encounter any issues? Thank you!

[hayashi-ay]

Hi @pkosiec, thank you for your efforts, but unfortunately, our app is still encountering issues with seeding, and we're receiving the following error message if password contain ?.

Start seeding...
  mongo-seeding Reading collections from /Users/ahayashi/app/workspaces/backend/seeder/data... +0ms
  mongo-seeding Transforming collections... +2s
  mongo-seeding Starting collection import... +1ms
TypeError: Cannot set property name of  which has only a getter
    at wrapError (/Users/ahayashi/app/node_modules/.pnpm/mongo-seeding@3.8.0-alpha.0/node_modules/mongo-seeding/dist/index.js:132:9)
    at Seeder.<anonymous> (/Users/ahayashi/app/node_modules/.pnpm/mongo-seeding@3.8.0-alpha.0/node_modules/mongo-seeding/dist/index.js:108:23)
    at Generator.throw (<anonymous>)
    at rejected (/Users/ahayashi/app/node_modules/.pnpm/mongo-seeding@3.8.0-alpha.0/node_modules/mongo-seeding/dist/index.js:20:64)
  mongo-seeding Reading collections from /Users/ahayashi/app/workspaces/backend/seeder/aeon/data... +0ms
  mongo-seeding Transforming collections... +49ms

Our app is performing seeding with source code like the following

    const seeder = new Seeder({
      database: {
        host: process.env.MONGO_HOST,
        port: Number(process.env.MONGO_PORT),
        name: 'e2e-test',
        username: process.env.MONGO_INITDB_ROOT_USERNAME,
        password: process.env.MONGO_INITDB_ROOT_PASSWORD,
        options: {
          authSource: 'admin',
          ...(process.env.AWS_DOCUMENTDB_SSL_CA_CERT && {
            tls: 'true',
            tlsCAFile: process.env.AWS_DOCUMENTDB_SSL_CA_CERT,
            replicaSet: 'rs0',
            readPreference: 'secondaryPreferred',
            retryWrites: 'false',
          }),
        },
      },
      dropCollections: true,
    });

    const collections = seeder.readCollectionsFromPath(
      path.resolve(__dirname, '../seeder/data'),
      {
        extensions: ['ts'],
        transformers: [
          Seeder.Transformers.setCreatedAtTimestamp,
          Seeder.Transformers.setUpdatedAtTimestamp,
        ],
      },
    );

    await seeder
      .import(collections)
      .then(() => {
        console.log('Done!');
      })
      .catch((err) => {
        console.log(err);
      });

[hayashi-ay]

We can perform seeding with mongo-seeding v3.7.2 by calling Seeder Class with an encoded password as a workaround. We're also using mongoose(@nestjs/mongoose) and migrate-mongo but neither of them requires client-side encoding . So there seems to be some difference and I suspect it may be related to whether the password is included in the uri or passed as an option when calling MongoClient.

const seeder = new Seeder({
      database: {
        host: process.env.MONGO_HOST,
        port: Number(process.env.MONGO_PORT),
        name: 'e2e-test',
        username: encodeURIComponent(process.env.MONGO_INITDB_ROOT_USERNAME),
        password: encodeURIComponent(process.env.MONGO_INITDB_ROOT_PASSWORD),
        options: {
          authSource: 'admin',
        },
      },
      dropCollections: true,
    });

[pkosiec]

Hey @hayashi-ay, this is how it is currently implemented in mongo-seeding: https://github.com/pkosiec/mongo-seeding/blob/main/core/src/database/database-connector.ts#L108

I'll take a look later at the projects you mentioned how they generate the URI based on the config properties 👍

[pkosiec]

Hey @hayashi-ay, thanks for your patience. Regarding the error TypeError: Cannot set property name of which has only a getter it is probably connected with some of the options you set. This is related to #211 and I've already fixed it.

Now, I got back to this problem with encoded password and trying to reproduce it with the latest version. ~It looks like the test case passes:~ ~(...)~ ~Are you sure you're encountering the password problem with the latest alpha? 🤔~

EDIT: Nvm, tested on actual MongoDB. Right, I need to use encodeURIComponent 👍

[pkosiec]

Good news @hayashi-ay @thundermiracle! Mongo Seeding 4.0.0 alpha is now ready to test: https://github.com/pkosiec/mongo-seeding/releases/tag/v4.0.0-alpha.0

Please let me know if it fixes your issues!