mongodb / specifications

Specifications related to MongoDB
http://specifications.readthedocs.io/en/latest
Other
389 stars 242 forks source link

DRIVERS-2524 do not create or drop `eccCollection` #1396

Closed kevinAlbs closed 1 year ago

kevinAlbs commented 1 year ago

Summary

Additional Improvements

Changes have been tested in the C driver here: https://github.com/mongodb/mongo-c-driver/pull/1232

Tests require libmongocrypt 1.8.0-alpha0 or newer. Binaries for 1.8.0-alpha0 are available here: https://spruce.mongodb.com/task/libmongocrypt_publish_upload_all_042603b1d72f49d9034e7059b82a65e843c7e38a_23_03_29_15_26_54/logs?execution=0

Background & Motivation

Terminology

Queryable Encryption is also referred to as QE or FLE2. The versions of the QE protocols are noted as QEv1 and QEv2. QEv1 was introduced in 6.0.0. QEv2 is introduced in 7.0.0.

Removal of eccCollection

This is requested by DRIVERS-2524. In QEv1, each QE collection required three additional metadata collections: eccCollection, ecocCollection, and escCollection. In QEv2, the eccCollection is no longer required.

Removal of documenting escCollection and ecocCollection

This is requested by DRIVERS-2586.

Addition of Wire Version check

QEv2 is a backwards breaking change to QEv1. Drivers supporting QEv2 will not support QEv1. Drivers supporting QEv1 will not support QEv2. QEv2 is introduced in server 7.0.0. I expect QEv2 will not be backported.

Sending an QEv1 payload to mongod 7.0.0-alpha-1139-gc9e3390 results in a server error. Example: (Location7292602) Encountered a Queryable Encryption find payload type that is no longer supported: 5

Sending a QEv2 payload to mongod 6.1.0 results in a server error. Example: Enumeration value '11' for field 'subtype' is not a valid value.

Creating a QE collection using encryptedFields without including eccCollection does not result in a server error from 6.1.0. Instead, the eccCollection is implicitly created by the server. The wire version check is intended to give users a helpful error if attempting to use a QEv2 driver on a QEv1 server. Without the wire version check, the user may not discover the incompatibility until a QEv2 payload is sent. This may result in the QE collections being created but being unusable.

Creating a collection with encryptedFields with eccCollection does not result in a server error from 7.0.0-alpha-1139-gc9e3390. SERVER-75683 requests returning a helpful error if attempting to create a QEv1 collection on a QEv2 server.

Interaction with createEncryptedCollection

The wire version check is proposed in createCollection. createEncryptedCollection calls createCollection after creating the data keys. If the wire version check fails, this may result in data keys being created and returned. This behavior is expected to be improved by the proposal in DRIVERS-2540 to create data keys and collections in a transaction.

Please complete the following before merging: