search

element-hq / element-meta

Shared/meta documentation and project artefacts for Element clients
75 stars 12 forks source link

Megolm Backup | Client should always have a local copy of all their megolm keys to limit the possibility of key loss #2446

Open BillCarsonFr opened 4 months ago

BillCarsonFr commented 4 months ago

We do not proactively download message keys from key storage (and instead, download keys on demand after a UTD).

This means that, if the user then resets their key backup (for example because they forgot their recovery key), it is likely that a significant number of message keys will be lost.

To address this, we will make each new client download (in the background) a copy of all the message keys in key storage as soon as it can. The operation should be resilient to app restart and should have a UI to report progress.

We'd rather not implement this twice, so our plan as of this writing is to switch Element Web to using the key-backup management code in matrix-rust-sdk, and then implement this once there.

Tasks:

Previous description ## Context We implemented a recent change to the way [we get the room keys from backup](https://github.com/element-hq/element-meta/issues/757). Previously, the full backup was downloaded after verification, as soon as the backup decryption key was known, but that did not result in a good user experience when your backup was too big. The time to download the full backup and decrypt it could be taking 10s of minutes depending on the platform. In order to improve that (and in the absence of [pagination](https://github.com/element-hq/element-meta/issues/2176) for the backup), keys are now downloaded on-demand when we try to decrypt an historical message. This results in a better user experience, as the messages get decrypted faster after a new login. ## Problem With this new way of getting the keys, all user sessions will now have only a partial set of the backed up keys. This can cause key loss if for example a user tries to reset his backup (without importing first from the previous backup). Notice that the previous implementation of full backup import has several issues: - [Performance](https://github.com/element-hq/element-web/issues/23359) - Lack of [pause/stop/resume](https://github.com/element-hq/element-web/issues/17950). Particularly on mobile, if the app was put in background and terminated by the OS before the end of import, it was not restarted. - Was possible to start several import in parallel ## Solution We want to ensure that each new client gets a copy of all the keys in backup as soon as it can. The operation should be resilient to app restart and should have an API [richvdh: or, more to the point: a UI?] to report progress. * EAX/EIX: - [ ] rust-sdk: Ensure that all the keys from a given backup version are downloaded - [ ] EIX: UI feedback for ongoing import - [ ] EAX: UI feedback for ongoing import - Web (https://github.com/element-hq/element-web/issues/27009): - [ ] js-sdk: Ensure that all the keys from a given backup version are downloaded - [ ] https://github.com/element-hq/element-web/issues/17839 - Element Android: - Element iOS ## Related/Existing issues - https://github.com/element-hq/element-web/issues/27009 - https://github.com/element-hq/element-web/issues/20694
richvdh commented 4 months ago

I wrote some notes on how we might do this for web at https://github.com/element-hq/element-web/issues/27009.

Notes:

richvdh commented 4 months ago

Question: is there any value in attempting to download individual keys on-demand once the full backup has been successfully downloaded? I'd think not?

andybalaam commented 2 weeks ago

@richvdh will remodel this item based on https://docs.google.com/document/d/1eEj_BagixDuny_fhF_WjoKEm0SD538mTXA3s436-c2I/edit - we chose the "Update Rust backup manager, and refactor EW to use it" option

BillCarsonFr commented 2 weeks ago

Question: is there any value in attempting to download individual keys on-demand once the full backup has been successfully downloaded? I'd think not?

Short answer: Once it has been successfully downloaded, no. But while it's beeing downoaladed definitly yes. If not it will take ages before I can see history after a login (unless we have that).

Longer answer: