Store sender data for Megolm sessions as per Invisible crypto

Part of Invisible Crypto.

Currently when we store an Inbound group session, we associate some SessionCreatorInfo to it. Currently this info mainly contains the curve25519 identity key of the session creator.

We need now to add more info:

user_id - the associated mxid of the session creator (rooted in Master Signing Key authenticity).
master_key - the Master Signing Key signing (via the SSK) this device at time of reception.
master_key_verified - true if at time of reception the user was verified (see here)

We are currently computing the "authenticity" of a megolm session but we are doing it at decryption time. The info we get at decryption time are sender: OwnedUserId and sender_device: OwnedDeviceId. Both related to a VerificationState. This owner info is untrusted (claimed), unless the VerificationState is Trusted.

The VerificationState variants are:

Verified
Unverified::UnverifiedIdentity
Unverified::UnsignedDevice
Unverified::MissingDevice
Unverified::InsecureSource

We are basically trying to get this same info but at reception time.

Three places we populate SenderData in an InboundGroupSession

When we receive the room keys via a to-device message. (This is already done via SenderDataFinder.)
When we get new or updated device info from /keys/query. To do this we need to be able to look up InboundGroupSessions that don't have SenderData by device curve key. This will require a new index on the inboundgroupsessions table. Question: if there are lots of sessions for a particular device, will we break everything by working through batches of them to update them?
When we decrypt a message for a session. In this case we have the session and device already, so it's just a case of persisting (into inboundgroupsessions) the VerificationState that we already look up at this moment.

The plan

In order to complete this we need to do these tasks:

Old tasks that are no longer relevant (see old plan in a comment below):

~~https://github.com/matrix-org/matrix-rust-sdk/issues/3545~~
~~https://github.com/matrix-org/matrix-rust-sdk/issues/3546~~
~~https://github.com/matrix-org/matrix-rust-sdk/issues/3547 (essentially just adds a next_retry_time so they get retried)~~

Out of scope

finish MSC4147
https://github.com/matrix-org/matrix-rust-sdk/issues/3548 . Requires an MSC.
https://github.com/matrix-org/matrix-rust-sdk/issues/3890

[Moved from https://github.com/element-hq/crypto-internal/issues/310]

Old description for reference:

Part of Invisible Crypto.

Currently when we store an Inbound group session, we associate some SessionCreatorInfo to it. Currently this info mainly contains the curve25519 identity key of the session creator.

We need now to add more info:

mxid The associated mxid of the session creator (rooted in MSK authenticity).
MSK The msk signing (via SSK) this device at time of reception.
MSK Trust Level at time of reception (TOFU|Verified, see here)

The VerificationState variants are:

Verified
Unverified::UnverifiedIdentity
Unverified::UnsignedDevice
Unverified::MissingDevice
Unverified::InsecureSource

We are basically trying to get this same info but at reception time.

Three speeds for handling a new session

When we receive a to-device message establishing a megolm session, we want to process it as quickly as possible, so there are 3 speeds:

Fast lane: if we already have the information we need in memory or the store, process it during the sync loop.
Slow lane: if we need to wait for Matrix API responses, don't block the sync loop. Instead, store the session in the store (in case we get restarted before we get a response) and kick off an async task that waits for the API response and processes the session later.
Background retry task: if the information we need is not immediately available from the Matrix API, store the session in the store and the periodic background/retry task will pick it up and retry it later.

"Quarantine"

(These ideas will be used by e.g. https://github.com/element-hq/crypto-internal/issues/308 - not really for this issue.)

In previous documents, we talked about "quarantine", but this is a difficult subject because different sessions will be acceptable to different clients at different times. We prefer to talk about what messages will be shown in what modes. These are some possible policies:

Show everything: show messages from all sessions, even if the device was not signed. This is likely to be the initial situation. Note: this excludes sessions where the cross-signing signature is incorrect: this is evidence of tampering so we discard those sessions.
Show only from cross-signed device + legacy: don't show messages that are non_legacy but have no verified sender info. Include session if:
- sender_data is of type SenderKnown, or
- legacy_session is true
Show only from cross-signed device: for environments where we expect no legacy sessions, we show only verified. Include session if:
- sender_data is of type SenderKnown
Top security ("paranoid"): for environments where TOFU trust is not good enough, we show only sessions from senders we have manually verified. Include session if:
- sender_data is of type SenderKnown, and
- msk_verified is true

Importing session keys

If we import a session from backup or from an exported file, we mark it as legacy and let the background task handle it.

The plan

In order to complete this we need to do these tasks:

https://github.com/matrix-org/matrix-rust-sdk/issues/3542
https://github.com/matrix-org/matrix-rust-sdk/issues/3543 (including adding a SessionManager to prevent clashes between concurrent tasks)
https://github.com/matrix-org/matrix-rust-sdk/issues/3545
https://github.com/matrix-org/matrix-rust-sdk/issues/3546
https://github.com/matrix-org/matrix-rust-sdk/issues/3547 (essentially just adds a next_retry_time so they get retried)

In addition, as a separate story, we need to this:

https://github.com/matrix-org/matrix-rust-sdk/issues/3548 . Requires an MSC.

[Moved from https://github.com/element-hq/crypto-internal/issues/310]

matrix-org / matrix-rust-sdk