richvdh
commented
9 months ago UX for this will be a labs button that causes the session to be restarted when it is enabled.
We probably need some sort of modal that indicates the state of the migration.
Closed richvdh closed 7 months ago
richvdh
commented
9 months ago UX for this will be a labs button that causes the session to be restarted when it is enabled.
We probably need some sort of modal that indicates the state of the migration.
richvdh
commented
8 months ago Some thoughts on how to achieve this:
[ ] The FFI bindings have a bunch of code to take libolm-pickled objects and store them in a matrix-rust-sdk CryptoStore. I think we need to push as much of this as we can down to matrix-sdk-crypto, and then expose it in the wasm bindings.
[ ] We need to add a bunch of methods to the legacy CryptoStore implementations (all three of them, though MemoryStore will be a no-op):
containsData: unlike the other methods, needs to be callable before startup. Checks if the database might contain data that needs migration (probably via the existing exists static methods).get/setMigrationState: stores/returns some integer which represents how many migration steps we have done (see below). We can probably stash the data itself in the account object. getEndToEndSessionsBatch, getEndToEndInboundGroupSessionsBatch: retrieve a batch of (pickled) Olm/Megolm sessions. getEndToEndInboundGroupSessionsBatch also needs to check sessions_needing_backup to see which sessions still need backup.deleteEndToEndSessionsBatch, deleteEndToEndInboundGroupSessionsBatch: delete a batch of Olm/Megolm sessions. (Presumably we pass them a list of sessions?)[ ] We make the "Rust cryptography implementation" button turn-on-able. (Currently, it is always disabled; change it to only be disabled when the feature is on; see RustCryptoSdkController.) We also make it reload the app when it is switched on; see ReloadOnChangeController for examples of how to do this. Some changes needed to the wording on the button.
This means that, once we restart, MatrixClientPeg.initClientCrypto will find useRustCrypto is true, causing it to call MatrixClient.initRustCrypto instead of MatrixClient.initCrypto. (This will be much the same codepath as when we drop legacy crypto support, which is good.)
[ ] Add a new migrateFromLegacyCrypto(legacyStore) method to js-sdk/src/rust-crypto, and call it in MatrixClient.initRustCrypto, before calling RustCrypto.initRustCrypto. First, it needs to check if there might be data to migrate:
CryptoStore.containsData and skip out if no data.CryptoStore.startup.CryptoStore.getMigrationState and skip any steps below that are done. Otherwise:CryptoStore.getSecretStorePrivateKey("master") etc)CryptoStore.getEndToEndRooms)CryptoStore.getSecretStorePrivateKey("m.megolm_backup.v1"))CryptoStore.setMigrationState to flag that this initial import has been done.We need to emit some sort of progress indication, so that the app can give feedback. Probably something as simple as a crypto.dataMigrationProgress event, whose payload includes two numbers including a total number of steps, and a number of steps so far. We can wire this into the progress_listener callback of the rust migration functions.
[ ] In MatrixChat.render, there is code for handling the state this.state.view === Views.LOGGED_IN && !this.state.ready. Refactor this to a new component. Have it listen for the crypto.dataMigrationProgress event, and when one is received, replace the spinner with some words and a ProgressBar.
We are ignoring:
session_problems - stores Olm sessions with decryption problems, for unwedging. We can let the Rust layer figure this out.Stuff we may need to figure out how to import later:
parked_shared_history and shared_history_inbound_group_sessons - something to do with history-share-on-invite, which we don't support in ER yet?richvdh
commented
8 months ago Looking at the performance of this as it stands:
On a test database containing 37 olm sessions and 4810 megolm sessions:
Obviously, that is dominated by the megolm session import, at 12.5s for 4810 sessions, or ~130ms for each batch of 50 sessions. The actual time for each batch is quite variable, but there are three stages:
Somewhat unexpected that deleting things from the old db is what takes the time. We could reduce it to (empirically) about 54.9ms by iterating through the store looking for rows to delete instead of using the index (which works because we know the batch will be at the "start" of the store); or we could rewrite the whole thing to store the progress made so far and delete everything on a single batch at the end. Not sure it is worthwhile though. If we need it to go faster, we can do it in the background.
At some point we will need to start actively migrating people who are using EW sessions on legacy Crypto to Rust crypto.
We will need a migration process for this. To gain confidence in the migration, we will add some sort of "developer feature" where people can request a migration. Later, we will automate the migration.