Mongoose Middleware and MongoDB CSFLE Explicit Encryption / Automatic Decryption

[harvey-sg]

Prerequisites

• [X] I have written a descriptive issue title

Mongoose version

6.5.2

Node.js version

18

MongoDB version

5.0.17

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

No response

Issue

Using the MongoDB Community Edition, and its CSFLE Explicit Encryption and Automatic Decryption, I am trying to figure out the best way to insert the ClientEncryption.encrypt() in the Mongoose “save” process.

Let’s say we want to encrypt the { lastName: String } field on save.

On one hand, the encrypted data must be saved on MongoDB as BinData (to benefit from automatic decryption for MongoDB), on the other: we would like to keep lastName defined as String in the Mongoose as schema.

Defining a field as String in the Mongoose schema forces the casting to String when saving the lastName (which would prevent the automatic decryption). The pre(‘save’) doesn’t seem to allow to recast the String to BinData.

To get it to work, we had to change the lastName field’s SchemaType from String to Mixed.

Here is the question: is there a way to keep the SchemaType as String, and invoke the ClientEncryption.encrypt() and recast the String to BinData just before Mongoose makes the call to the MongoDB? Maybe using a specific hook in the Mongoose middleware?

Thank you for your suggestion.

[vkarpov15]

You would need to set the autoEncryption.schemaMap property to mongoose.connect(), here's more info in the mongodb node driver docs and some info on how to set up field level encryption with Node and MongoDB.

Mongoose itself doesn't handle field level encryption, that is the responsibility of the MongoDB Node driver. You have 2 ways to define what fields to encrypt/decrypt:

1) Using schemaMap option to mongoose.connect()

'use strict';

const { ClientEncryption } = require('mongodb-client-encryption');
const mongoose = require('mongoose');
const { Binary } = require('mongodb');

run().catch(err => console.log(err));

async function run() {
  /* Step 1: Connect to MongoDB and insert a key */

  // Create a very basic key. You're responsible for making
  // your key secure, don't use this in prod :)
  const arr = [];
  for (let i = 0; i < 96; ++i) {
    arr.push(i);
  }
  const key = Buffer.from(arr);

  const keyVaultNamespace = 'client.encryption';
  const kmsProviders = { local: { key } };

  const conn = await mongoose.createConnection('mongodb://127.0.0.1:27017/mongoose_test', {
    autoEncryption: {
      keyVaultNamespace,
      kmsProviders
    }
  }).asPromise();
  const encryption = new ClientEncryption(conn.client, {
    keyVaultNamespace,
    kmsProviders,
  });

  const _key = await encryption.createDataKey('local');

  /* Step 2: connect using schema map and new key */
  await mongoose.connect('mongodb://127.0.0.1:27017/mongoose_test', {
    // Configure auto encryption
    autoEncryption: {
      keyVaultNamespace,
      kmsProviders,
      schemaMap: {
        'mongoose_test.tests': {
          bsonType: 'object',
          encryptMetadata: {
            keyId: [_key]
          },
          properties: {
            name: {
              encrypt: {
                bsonType: 'string',
                algorithm: 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic'
              }
            }
          }
        }
      }
    }
  });

  /* Step 2: Create a Mongoose model that uses the encrypted collection and see encryption in action */

  // 'super secret' will be stored as 'BinData' in the database,
  // if you query using the `mongo` shell.
  const Model = mongoose.model('Test', mongoose.Schema({ name: String }));
  await Model.create({ name: 'super secret' });

  console.log('Done');
}

2) Using a JSON schema validator on your collection:

'use strict';

const { ClientEncryption } = require('mongodb-client-encryption');
const base64 = require('uuid-base64');
const mongoose = require('mongoose');

run().catch(err => console.log(err));

async function run() {
  /* Step 1: Connect to MongoDB using autoEncryption */

  // Create a very basic key. You're responsible for making
  // your key secure, don't use this in prod :)
  const arr = [];
  for (let i = 0; i < 96; ++i) {
    arr.push(i);
  }
  const key = Buffer.from(arr);

  const keyVaultNamespace = 'client.encryption';
  const kmsProviders = { local: { key } };
  await mongoose.connect('ATLAS URI HERE', {
    // Configure auto encryption
    autoEncryption: {
      keyVaultNamespace,
      kmsProviders
    }
  });

    /* Step 2: create a key and configure encryption using JSONschema */

  const encryption = new ClientEncryption(mongoose.connection.client, {
      keyVaultNamespace,
      kmsProviders,
    });

  const _key = await encryption.createDataKey('local');

    // CSFLE is defined through JSON schema. Easiest way to set
  // a JSON schema on your collection is via `createCollection()`
  await mongoose.connection.dropCollection('tests').catch(() => {});
  await mongoose.connection.createCollection('tests', {
    validator: {
      $jsonSchema: {
        bsonType: 'object',
        properties: {
          // Automatically encrypt the 'name' property
          name: {
            encrypt: {
              bsonType: 'string',
              keyId: [_key],
              algorithm: 'AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic' }
          }
        }
      }
    }
  });

  /* Step 3: Create a Mongoose model that uses the encrypted collection and see encryption in action */

  // 'super secret' will be stored as 'BinData' in the database,
  // if you query using the `mongo` shell.
  const Model = mongoose.model('Test', mongoose.Schema({ name: String }));
  await Model.create({ name: 'super secret' });

  console.log('Done');
}

[harvey-sg]

Hi @vkarpov15 , thank you for your response. It seems that your answer is relevant to the automatic encryption (which is not available in the MongoDB Community Edition). In our case, we use the explicit encryption, that requires the use of ClientEncryption.encrypt() and a mongoDB connection with bypassAutoEncryption: true. Here is some info about this use case.

    autoEncryption: {
      keyVaultNamespace,
      kmsProviders,
      bypassAutoEncryption: true
    }

Our current solution works (explicit encryption/automatic decryption) if we use Mixed as a schema type.

[muhammadmubeen12345]

I've tried different methods, but upon examining the database, I observed that the data is stored as a string rather than binary. This deviation is causing problems with the decryption process. Upon reviewing the MongoDB driver documentation, it appears that data must be saved in BinaryData.

[ben12qi]

It should be noted the original poster's version was prior to explicit encryption (i.e. not automatic) being possible with community edition but this was released for public preview in 6.2 and on Aug 15, 2023 it was released as part of version 7 Community Edition. The relevant documentation is here:

https://www.mongodb.com/docs/v6.2/core/queryable-encryption/tutorials/explicit-encryption/ (for comparison purposes) https://www.mongodb.com/docs/v7.0/core/queryable-encryption/tutorials/explicit-encryption/ (current version)

It looks like Mongoose is incompatible for this new scenario and it is likely a new feature request.

[vkarpov15]

It looks like explicit encryption works reasonably fine in Mongoose. Below is sample code using explicit encryption in Mongoose 7 and mongodb-client-encryption@2. I can't say the below DX is great; you need to explicitly define encrypted fields as being of type 'Buffer', and you need to explicitly encrypt and decrypt because Mongoose doesn't support async getters/setters.

'use strict';

const { ClientEncryption } = require('mongodb-client-encryption');
const mongoose = require('mongoose');

mongoose.set('debug', true);

run().catch(err => console.log(err));

async function run() {
  // Create a very basic key. You're responsible for making
  // your key secure, don't use this in prod :)
  const arr = [];
  for (let i = 0; i < 96; ++i) {
    arr.push(i);
  }
  const key = Buffer.from(arr);

  const keyVaultNamespace = 'client.encryption';
  const kmsProviders = { local: { key } };

  const conn = await mongoose.createConnection('mongodb://127.0.0.1:27017/mongoose_test', {
  }).asPromise();
  const encryption = new ClientEncryption(conn.client, {
    keyVaultNamespace,
    kmsProviders,
  });

  const dataKeyId = await encryption.createDataKey('local');

  await mongoose.connect('mongodb://127.0.0.1:27017/mongoose_test');

  // 'super secret' will be stored as 'BinData' in the database,
  // if you query using the `mongo` shell.
  const Model = mongoose.model('Test', mongoose.Schema({
    name: {
      type: Buffer
    }
  }));

  const encryptedName = await encryption.encrypt('super secret', {
    algorithm: "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic",
    keyId: dataKeyId,
  });

  const doc = await Model.create({ name: encryptedName });
  console.log(
    await encryption.decrypt(
      doc.name
    )
  );

  console.log('Done');
}

Could be better, we will leave this open to consider better explicit encryption support for future work. Especially if we reconsider support for async getters/setters. But at least the above should confirm that you can implement explicit encryption with Mongoose.